Pay Attention to the Ending:Strong Neural Baselines for the ROC Story Cloze Task

Zheng Cai, Lifu Tu, Kevin Gimpel


Abstract
We consider the ROC story cloze task (Mostafazadeh et al., 2016) and present several findings. We develop a model that uses hierarchical recurrent networks with attention to encode the sentences in the story and score candidate endings. By discarding the large training set and only training on the validation set, we achieve an accuracy of 74.7%. Even when we discard the story plots (sentences before the ending) and only train to choose the better of two endings, we can still reach 72.5%. We then analyze this “ending-only” task setting. We estimate human accuracy to be 78% and find several types of clues that lead to this high accuracy, including those related to sentiment, negation, and general ending likelihood regardless of the story context.
Anthology ID:
P17-2097
Volume:
Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers)
Month:
July
Year:
2017
Address:
Vancouver, Canada
Editors:
Regina Barzilay, Min-Yen Kan
Venue:
ACL
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
616–622
Language:
URL:
https://aclanthology.org/P17-2097
DOI:
10.18653/v1/P17-2097
Bibkey:
Cite (ACL):
Zheng Cai, Lifu Tu, and Kevin Gimpel. 2017. Pay Attention to the Ending:Strong Neural Baselines for the ROC Story Cloze Task. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 616–622, Vancouver, Canada. Association for Computational Linguistics.
Cite (Informal):
Pay Attention to the Ending:Strong Neural Baselines for the ROC Story Cloze Task (Cai et al., ACL 2017)
Copy Citation:
PDF:
https://aclanthology.org/P17-2097.pdf