Incremental Transformer with Deliberation Decoder for Document Grounded Conversations

Zekang Li, Cheng Niu, Fandong Meng, Yang Feng, Qian Li, Jie Zhou


Abstract
Document Grounded Conversations is a task to generate dialogue responses when chatting about the content of a given document. Obviously, document knowledge plays a critical role in Document Grounded Conversations, while existing dialogue models do not exploit this kind of knowledge effectively enough. In this paper, we propose a novel Transformer-based architecture for multi-turn document grounded conversations. In particular, we devise an Incremental Transformer to encode multi-turn utterances along with knowledge in related documents. Motivated by the human cognitive process, we design a two-pass decoder (Deliberation Decoder) to improve context coherence and knowledge correctness. Our empirical study on a real-world Document Grounded Dataset proves that responses generated by our model significantly outperform competitive baselines on both context coherence and knowledge relevance.
Anthology ID:
P19-1002
Volume:
Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics
Month:
July
Year:
2019
Address:
Florence, Italy
Editors:
Anna Korhonen, David Traum, Lluís Màrquez
Venue:
ACL
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
12–21
Language:
URL:
https://aclanthology.org/P19-1002
DOI:
10.18653/v1/P19-1002
Bibkey:
Cite (ACL):
Zekang Li, Cheng Niu, Fandong Meng, Yang Feng, Qian Li, and Jie Zhou. 2019. Incremental Transformer with Deliberation Decoder for Document Grounded Conversations. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 12–21, Florence, Italy. Association for Computational Linguistics.
Cite (Informal):
Incremental Transformer with Deliberation Decoder for Document Grounded Conversations (Li et al., ACL 2019)
Copy Citation:
PDF:
https://aclanthology.org/P19-1002.pdf
Video:
 https://aclanthology.org/P19-1002.mp4
Code
 lizekang/ITDD +  additional community code
Data
CMU DoG