Vectors or Graphs? On Differences of Representations for Distributional Semantic Models

Chris Biemann


Abstract
Distributional Semantic Models (DSMs) have recently received increased attention, together with the rise of neural architectures for scalable training of dense vector embeddings. While some of the literature even includes terms like ‘vectors’ and ‘dimensionality’ in the definition of DSMs, there are some good reasons why we should consider alternative formulations of distributional models. As an instance, I present a scalable graph-based solution to distributional semantics. The model belongs to the family of ‘count-based’ DSMs, keeps its representation sparse and explicit, and thus fully interpretable. I will highlight some important differences between sparse graph-based and dense vector approaches to DSMs: while dense vector-based models are computationally easier to handle and provide a nice uniform representation that can be compared and combined in many ways, they lack interpretability, provenance and robustness. On the other hand, graph-based sparse models have a more straightforward interpretation, handle sense distinctions more naturally and can straightforwardly be linked to knowledge bases, while lacking the ability to compare arbitrary lexical units and a compositionality operation. Since both representations have their merits, I opt for exploring their combination in the outlook.
Anthology ID:
W16-5301
Volume:
Proceedings of the 5th Workshop on Cognitive Aspects of the Lexicon (CogALex - V)
Month:
December
Year:
2016
Address:
Osaka, Japan
Editors:
Michael Zock, Alessandro Lenci, Stefan Evert
Venue:
CogALex
SIG:
SIGLEX
Publisher:
The COLING 2016 Organizing Committee
Note:
Pages:
1–7
Language:
URL:
https://aclanthology.org/W16-5301
DOI:
Bibkey:
Cite (ACL):
Chris Biemann. 2016. Vectors or Graphs? On Differences of Representations for Distributional Semantic Models. In Proceedings of the 5th Workshop on Cognitive Aspects of the Lexicon (CogALex - V), pages 1–7, Osaka, Japan. The COLING 2016 Organizing Committee.
Cite (Informal):
Vectors or Graphs? On Differences of Representations for Distributional Semantic Models (Biemann, CogALex 2016)
Copy Citation:
PDF:
https://aclanthology.org/W16-5301.pdf