An Analysis of Attention over Clinical Notes for Predictive Tasks

Sarthak Jain, Ramin Mohammadi, Byron C. Wallace


Abstract
The shift to electronic medical records (EMRs) has engendered research into machine learning and natural language technologies to analyze patient records, and to predict from these clinical outcomes of interest. Two observations motivate our aims here. First, unstructured notes contained within EMR often contain key information, and hence should be exploited by models. Second, while strong predictive performance is important, interpretability of models is perhaps equally so for applications in this domain. Together, these points suggest that neural models for EMR may benefit from incorporation of attention over notes, which one may hope will both yield performance gains and afford transparency in predictions. In this work we perform experiments to explore this question using two EMR corpora and four different predictive tasks, that: (i) inclusion of attention mechanisms is critical for neural encoder modules that operate over notes fields in order to yield competitive performance, but, (ii) unfortunately, while these boost predictive performance, it is decidedly less clear whether they provide meaningful support for predictions.
Anthology ID:
W19-1902
Volume:
Proceedings of the 2nd Clinical Natural Language Processing Workshop
Month:
June
Year:
2019
Address:
Minneapolis, Minnesota, USA
Editors:
Anna Rumshisky, Kirk Roberts, Steven Bethard, Tristan Naumann
Venue:
ClinicalNLP
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
15–21
Language:
URL:
https://aclanthology.org/W19-1902
DOI:
10.18653/v1/W19-1902
Bibkey:
Cite (ACL):
Sarthak Jain, Ramin Mohammadi, and Byron C. Wallace. 2019. An Analysis of Attention over Clinical Notes for Predictive Tasks. In Proceedings of the 2nd Clinical Natural Language Processing Workshop, pages 15–21, Minneapolis, Minnesota, USA. Association for Computational Linguistics.
Cite (Informal):
An Analysis of Attention over Clinical Notes for Predictive Tasks (Jain et al., ClinicalNLP 2019)
Copy Citation:
PDF:
https://aclanthology.org/W19-1902.pdf