Are We Consistently Biased? Multidimensional Analysis of Biases in Distributional Word Vectors

Anne Lauscher, Goran Glavaš


Abstract
Word embeddings have recently been shown to reflect many of the pronounced societal biases (e.g., gender bias or racial bias). Existing studies are, however, limited in scope and do not investigate the consistency of biases across relevant dimensions like embedding models, types of texts, and different languages. In this work, we present a systematic study of biases encoded in distributional word vector spaces: we analyze how consistent the bias effects are across languages, corpora, and embedding models. Furthermore, we analyze the cross-lingual biases encoded in bilingual embedding spaces, indicative of the effects of bias transfer encompassed in cross-lingual transfer of NLP models. Our study yields some unexpected findings, e.g., that biases can be emphasized or downplayed by different embedding models or that user-generated content may be less biased than encyclopedic text. We hope our work catalyzes bias research in NLP and informs the development of bias reduction techniques.
Anthology ID:
S19-1010
Volume:
Proceedings of the Eighth Joint Conference on Lexical and Computational Semantics (*SEM 2019)
Month:
June
Year:
2019
Address:
Minneapolis, Minnesota
Editors:
Rada Mihalcea, Ekaterina Shutova, Lun-Wei Ku, Kilian Evang, Soujanya Poria
Venue:
*SEM
SIGs:
SIGLEX | SIGSEM
Publisher:
Association for Computational Linguistics
Note:
Pages:
85–91
Language:
URL:
https://aclanthology.org/S19-1010
DOI:
10.18653/v1/S19-1010
Bibkey:
Cite (ACL):
Anne Lauscher and Goran Glavaš. 2019. Are We Consistently Biased? Multidimensional Analysis of Biases in Distributional Word Vectors. In Proceedings of the Eighth Joint Conference on Lexical and Computational Semantics (*SEM 2019), pages 85–91, Minneapolis, Minnesota. Association for Computational Linguistics.
Cite (Informal):
Are We Consistently Biased? Multidimensional Analysis of Biases in Distributional Word Vectors (Lauscher & Glavaš, *SEM 2019)
Copy Citation:
PDF:
https://aclanthology.org/S19-1010.pdf
Code
 umanlp/XWEAT