Dynamic Feature Selection with Attention in Incremental Parsing

Ryosuke Kohita, Hiroshi Noji, Yuji Matsumoto


Abstract
One main challenge for incremental transition-based parsers, when future inputs are invisible, is to extract good features from a limited local context. In this work, we present a simple technique to maximally utilize the local features with an attention mechanism, which works as context- dependent dynamic feature selection. Our model learns, for example, which tokens should a parser focus on, to decide the next action. Our multilingual experiment shows its effectiveness across many languages. We also present an experiment with augmented test dataset and demon- strate it helps to understand the model’s behavior on locally ambiguous points.
Anthology ID:
C18-1067
Volume:
Proceedings of the 27th International Conference on Computational Linguistics
Month:
August
Year:
2018
Address:
Santa Fe, New Mexico, USA
Editors:
Emily M. Bender, Leon Derczynski, Pierre Isabelle
Venue:
COLING
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
785–794
Language:
URL:
https://aclanthology.org/C18-1067
DOI:
Bibkey:
Cite (ACL):
Ryosuke Kohita, Hiroshi Noji, and Yuji Matsumoto. 2018. Dynamic Feature Selection with Attention in Incremental Parsing. In Proceedings of the 27th International Conference on Computational Linguistics, pages 785–794, Santa Fe, New Mexico, USA. Association for Computational Linguistics.
Cite (Informal):
Dynamic Feature Selection with Attention in Incremental Parsing (Kohita et al., COLING 2018)
Copy Citation:
PDF:
https://aclanthology.org/C18-1067.pdf