Deep Pivot-Based Modeling for Cross-language Cross-domain Transfer with Minimal Guidance

Yftah Ziser, Roi Reichart


Abstract
While cross-domain and cross-language transfer have long been prominent topics in NLP research, their combination has hardly been explored. In this work we consider this problem, and propose a framework that builds on pivot-based learning, structure-aware Deep Neural Networks (particularly LSTMs and CNNs) and bilingual word embeddings, with the goal of training a model on labeled data from one (language, domain) pair so that it can be effectively applied to another (language, domain) pair. We consider two setups, differing with respect to the unlabeled data available for model training. In the full setup the model has access to unlabeled data from both pairs, while in the lazy setup, which is more realistic for truly resource-poor languages, unlabeled data is available for both domains but only for the source language. We design our model for the lazy setup so that for a given target domain, it can train once on the source language and then be applied to any target language without re-training. In experiments with nine English-German and nine English-French domain pairs our best model substantially outperforms previous models even when it is trained in the lazy setup and previous models are trained in the full setup.
Anthology ID:
D18-1022
Volume:
Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing
Month:
October-November
Year:
2018
Address:
Brussels, Belgium
Editors:
Ellen Riloff, David Chiang, Julia Hockenmaier, Jun’ichi Tsujii
Venue:
EMNLP
SIG:
SIGDAT
Publisher:
Association for Computational Linguistics
Note:
Pages:
238–249
Language:
URL:
https://aclanthology.org/D18-1022
DOI:
10.18653/v1/D18-1022
Bibkey:
Cite (ACL):
Yftah Ziser and Roi Reichart. 2018. Deep Pivot-Based Modeling for Cross-language Cross-domain Transfer with Minimal Guidance. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 238–249, Brussels, Belgium. Association for Computational Linguistics.
Cite (Informal):
Deep Pivot-Based Modeling for Cross-language Cross-domain Transfer with Minimal Guidance (Ziser & Reichart, EMNLP 2018)
Copy Citation:
PDF:
https://aclanthology.org/D18-1022.pdf
Code
 yftah89/PBLM-Cross-language-Cross-domain