A Multilingual BPE Embedding Space for Universal Sentiment Lexicon Induction

Mengjie Zhao, Hinrich Schütze


Abstract
We present a new method for sentiment lexicon induction that is designed to be applicable to the entire range of typological diversity of the world’s languages. We evaluate our method on Parallel Bible Corpus+ (PBC+), a parallel corpus of 1593 languages. The key idea is to use Byte Pair Encodings (BPEs) as basic units for multilingual embeddings. Through zero-shot transfer from English sentiment, we learn a seed lexicon for each language in the domain of PBC+. Through domain adaptation, we then generalize the domain-specific lexicon to a general one. We show – across typologically diverse languages in PBC+ – good quality of seed and general-domain sentiment lexicons by intrinsic and extrinsic and by automatic and human evaluation. We make freely available our code, seed sentiment lexicons for all 1593 languages and induced general-domain sentiment lexicons for 200 languages.
Anthology ID:
P19-1341
Volume:
Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics
Month:
July
Year:
2019
Address:
Florence, Italy
Editors:
Anna Korhonen, David Traum, Lluís Màrquez
Venue:
ACL
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
3506–3517
Language:
URL:
https://aclanthology.org/P19-1341
DOI:
10.18653/v1/P19-1341
Bibkey:
Cite (ACL):
Mengjie Zhao and Hinrich Schütze. 2019. A Multilingual BPE Embedding Space for Universal Sentiment Lexicon Induction. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 3506–3517, Florence, Italy. Association for Computational Linguistics.
Cite (Informal):
A Multilingual BPE Embedding Space for Universal Sentiment Lexicon Induction (Zhao & Schütze, ACL 2019)
Copy Citation:
PDF:
https://aclanthology.org/P19-1341.pdf
Video:
 https://aclanthology.org/P19-1341.mp4