Exploiting Attention to Reveal Shortcomings in Memory Models

Kaylee Burns, Aida Nematzadeh, Erin Grant, Alison Gopnik, Tom Griffiths


Abstract
The decision making processes of deep networks are difficult to understand and while their accuracy often improves with increased architectural complexity, so too does their opacity. Practical use of machine learning models, especially for question and answering applications, demands a system that is interpretable. We analyze the attention of a memory network model to reconcile contradictory performance on a challenging question-answering dataset that is inspired by theory-of-mind experiments. We equate success on questions to task classification, which explains not only test-time failures but also how well the model generalizes to new training conditions.
Anthology ID:
W18-5454
Volume:
Proceedings of the 2018 EMNLP Workshop BlackboxNLP: Analyzing and Interpreting Neural Networks for NLP
Month:
November
Year:
2018
Address:
Brussels, Belgium
Editors:
Tal Linzen, Grzegorz Chrupała, Afra Alishahi
Venue:
EMNLP
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
378–380
Language:
URL:
https://aclanthology.org/W18-5454
DOI:
10.18653/v1/W18-5454
Bibkey:
Cite (ACL):
Kaylee Burns, Aida Nematzadeh, Erin Grant, Alison Gopnik, and Tom Griffiths. 2018. Exploiting Attention to Reveal Shortcomings in Memory Models. In Proceedings of the 2018 EMNLP Workshop BlackboxNLP: Analyzing and Interpreting Neural Networks for NLP, pages 378–380, Brussels, Belgium. Association for Computational Linguistics.
Cite (Informal):
Exploiting Attention to Reveal Shortcomings in Memory Models (Burns et al., EMNLP 2018)
Copy Citation:
PDF:
https://aclanthology.org/W18-5454.pdf