Automatic Selection of Context Configurations for Improved Class-Specific Word Representations

Ivan Vulić, Roy Schwartz, Ari Rappoport, Roi Reichart, Anna Korhonen


Abstract
This paper is concerned with identifying contexts useful for training word representation models for different word classes such as adjectives (A), verbs (V), and nouns (N). We introduce a simple yet effective framework for an automatic selection of class-specific context configurations. We construct a context configuration space based on universal dependency relations between words, and efficiently search this space with an adapted beam search algorithm. In word similarity tasks for each word class, we show that our framework is both effective and efficient. Particularly, it improves the Spearman’s rho correlation with human scores on SimLex-999 over the best previously proposed class-specific contexts by 6 (A), 6 (V) and 5 (N) rho points. With our selected context configurations, we train on only 14% (A), 26.2% (V), and 33.6% (N) of all dependency-based contexts, resulting in a reduced training time. Our results generalise: we show that the configurations our algorithm learns for one English training setup outperform previously proposed context types in another training setup for English. Moreover, basing the configuration space on universal dependencies, it is possible to transfer the learned configurations to German and Italian. We also demonstrate improved per-class results over other context types in these two languages..
Anthology ID:
K17-1013
Volume:
Proceedings of the 21st Conference on Computational Natural Language Learning (CoNLL 2017)
Month:
August
Year:
2017
Address:
Vancouver, Canada
Editors:
Roger Levy, Lucia Specia
Venue:
CoNLL
SIG:
SIGNLL
Publisher:
Association for Computational Linguistics
Note:
Pages:
112–122
Language:
URL:
https://aclanthology.org/K17-1013
DOI:
10.18653/v1/K17-1013
Bibkey:
Cite (ACL):
Ivan Vulić, Roy Schwartz, Ari Rappoport, Roi Reichart, and Anna Korhonen. 2017. Automatic Selection of Context Configurations for Improved Class-Specific Word Representations. In Proceedings of the 21st Conference on Computational Natural Language Learning (CoNLL 2017), pages 112–122, Vancouver, Canada. Association for Computational Linguistics.
Cite (Informal):
Automatic Selection of Context Configurations for Improved Class-Specific Word Representations (Vulić et al., CoNLL 2017)
Copy Citation:
PDF:
https://aclanthology.org/K17-1013.pdf