SwitchOut: an Efficient Data Augmentation Algorithm for Neural Machine Translation

Xinyi Wang, Hieu Pham, Zihang Dai, Graham Neubig


Abstract
In this work, we examine methods for data augmentation for text-based tasks such as neural machine translation (NMT). We formulate the design of a data augmentation policy with desirable properties as an optimization problem, and derive a generic analytic solution. This solution not only subsumes some existing augmentation schemes, but also leads to an extremely simple data augmentation strategy for NMT: randomly replacing words in both the source sentence and the target sentence with other random words from their corresponding vocabularies. We name this method SwitchOut. Experiments on three translation datasets of different scales show that SwitchOut yields consistent improvements of about 0.5 BLEU, achieving better or comparable performances to strong alternatives such as word dropout (Sennrich et al., 2016a). Code to implement this method is included in the appendix.
Anthology ID:
D18-1100
Volume:
Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing
Month:
October-November
Year:
2018
Address:
Brussels, Belgium
Editors:
Ellen Riloff, David Chiang, Julia Hockenmaier, Jun’ichi Tsujii
Venue:
EMNLP
SIG:
SIGDAT
Publisher:
Association for Computational Linguistics
Note:
Pages:
856–861
Language:
URL:
https://aclanthology.org/D18-1100
DOI:
10.18653/v1/D18-1100
Bibkey:
Cite (ACL):
Xinyi Wang, Hieu Pham, Zihang Dai, and Graham Neubig. 2018. SwitchOut: an Efficient Data Augmentation Algorithm for Neural Machine Translation. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 856–861, Brussels, Belgium. Association for Computational Linguistics.
Cite (Informal):
SwitchOut: an Efficient Data Augmentation Algorithm for Neural Machine Translation (Wang et al., EMNLP 2018)
Copy Citation:
PDF:
https://aclanthology.org/D18-1100.pdf
Attachment:
 D18-1100.Attachment.pdf
Video:
 https://vimeo.com/305206127