Unsupervised Source Hierarchies for Low-Resource Neural Machine Translation

Anna Currey, Kenneth Heafield


Abstract
Incorporating source syntactic information into neural machine translation (NMT) has recently proven successful (Eriguchi et al., 2016; Luong et al., 2016). However, this is generally done using an outside parser to syntactically annotate the training data, making this technique difficult to use for languages or domains for which a reliable parser is not available. In this paper, we introduce an unsupervised tree-to-sequence (tree2seq) model for neural machine translation; this model is able to induce an unsupervised hierarchical structure on the source sentence based on the downstream task of neural machine translation. We adapt the Gumbel tree-LSTM of Choi et al. (2018) to NMT in order to create the encoder. We evaluate our model against sequential and supervised parsing baselines on three low- and medium-resource language pairs. For low-resource cases, the unsupervised tree2seq encoder significantly outperforms the baselines; no improvements are seen for medium-resource translation.
Anthology ID:
W18-2902
Volume:
Proceedings of the Workshop on the Relevance of Linguistic Structure in Neural Architectures for NLP
Month:
July
Year:
2018
Address:
Melbourne, Australia
Editors:
Georgiana Dinu, Miguel Ballesteros, Avirup Sil, Sam Bowman, Wael Hamza, Anders Sogaard, Tahira Naseem, Yoav Goldberg
Venue:
ACL
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
6–12
Language:
URL:
https://aclanthology.org/W18-2902
DOI:
10.18653/v1/W18-2902
Bibkey:
Cite (ACL):
Anna Currey and Kenneth Heafield. 2018. Unsupervised Source Hierarchies for Low-Resource Neural Machine Translation. In Proceedings of the Workshop on the Relevance of Linguistic Structure in Neural Architectures for NLP, pages 6–12, Melbourne, Australia. Association for Computational Linguistics.
Cite (Informal):
Unsupervised Source Hierarchies for Low-Resource Neural Machine Translation (Currey & Heafield, ACL 2018)
Copy Citation:
PDF:
https://aclanthology.org/W18-2902.pdf