Convolution-Enhanced Bilingual Recursive Neural Network for Bilingual Semantic Modeling

Jinsong Su, Biao Zhang, Deyi Xiong, Ruochen Li, Jianmin Yin


Abstract
Estimating similarities at different levels of linguistic units, such as words, sub-phrases and phrases, is helpful for measuring semantic similarity of an entire bilingual phrase. In this paper, we propose a convolution-enhanced bilingual recursive neural network (ConvBRNN), which not only exploits word alignments to guide the generation of phrase structures but also integrates multiple-level information of the generated phrase structures into bilingual semantic modeling. In order to accurately learn the semantic hierarchy of a bilingual phrase, we develop a recursive neural network to constrain the learned bilingual phrase structures to be consistent with word alignments. Upon the generated source and target phrase structures, we stack a convolutional neural network to integrate vector representations of linguistic units on the structures into bilingual phrase embeddings. After that, we fully incorporate information of different linguistic units into a bilinear semantic similarity model. We introduce two max-margin losses to train the ConvBRNN model: one for the phrase structure inference and the other for the semantic similarity model. Experiments on NIST Chinese-English translation tasks demonstrate the high quality of the generated bilingual phrase structures with respect to word alignments and the effectiveness of learned semantic similarities on machine translation.
Anthology ID:
C16-1289
Volume:
Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers
Month:
December
Year:
2016
Address:
Osaka, Japan
Editors:
Yuji Matsumoto, Rashmi Prasad
Venue:
COLING
SIG:
Publisher:
The COLING 2016 Organizing Committee
Note:
Pages:
3071–3081
Language:
URL:
https://aclanthology.org/C16-1289
DOI:
Bibkey:
Cite (ACL):
Jinsong Su, Biao Zhang, Deyi Xiong, Ruochen Li, and Jianmin Yin. 2016. Convolution-Enhanced Bilingual Recursive Neural Network for Bilingual Semantic Modeling. In Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers, pages 3071–3081, Osaka, Japan. The COLING 2016 Organizing Committee.
Cite (Informal):
Convolution-Enhanced Bilingual Recursive Neural Network for Bilingual Semantic Modeling (Su et al., COLING 2016)
Copy Citation:
PDF:
https://aclanthology.org/C16-1289.pdf