Quantifying Context Overlap for Training Word Embeddings

Yimeng Zhuang, Jinghui Xie, Yinhe Zheng, Xuan Zhu


Abstract
Most models for learning word embeddings are trained based on the context information of words, more precisely first order co-occurrence relations. In this paper, a metric is designed to estimate second order co-occurrence relations based on context overlap. The estimated values are further used as the augmented data to enhance the learning of word embeddings by joint training with existing neural word embedding models. Experimental results show that better word vectors can be obtained for word similarity tasks and some downstream NLP tasks by the enhanced approach.
Anthology ID:
D18-1057
Volume:
Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing
Month:
October-November
Year:
2018
Address:
Brussels, Belgium
Editors:
Ellen Riloff, David Chiang, Julia Hockenmaier, Jun’ichi Tsujii
Venue:
EMNLP
SIG:
SIGDAT
Publisher:
Association for Computational Linguistics
Note:
Pages:
587–593
Language:
URL:
https://aclanthology.org/D18-1057
DOI:
10.18653/v1/D18-1057
Bibkey:
Cite (ACL):
Yimeng Zhuang, Jinghui Xie, Yinhe Zheng, and Xuan Zhu. 2018. Quantifying Context Overlap for Training Word Embeddings. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 587–593, Brussels, Belgium. Association for Computational Linguistics.
Cite (Informal):
Quantifying Context Overlap for Training Word Embeddings (Zhuang et al., EMNLP 2018)
Copy Citation:
PDF:
https://aclanthology.org/D18-1057.pdf
Video:
 https://vimeo.com/305196755
Data
SST