Using Reinforcement Learning to Model Incrementality in a Fast-Paced Dialogue Game

Ramesh Manuvinakurike, David DeVault, Kallirroi Georgila


Abstract
We apply Reinforcement Learning (RL) to the problem of incremental dialogue policy learning in the context of a fast-paced dialogue game. We compare the policy learned by RL with a high-performance baseline policy which has been shown to perform very efficiently (nearly as well as humans) in this dialogue game. The RL policy outperforms the baseline policy in offline simulations (based on real user data). We provide a detailed comparison of the RL policy and the baseline policy, including information about how much effort and time it took to develop each one of them. We also highlight the cases where the RL policy performs better, and show that understanding the RL policy can provide valuable insights which can inform the creation of an even better rule-based policy.
Anthology ID:
W17-5539
Volume:
Proceedings of the 18th Annual SIGdial Meeting on Discourse and Dialogue
Month:
August
Year:
2017
Address:
Saarbrücken, Germany
Editors:
Kristiina Jokinen, Manfred Stede, David DeVault, Annie Louis
Venue:
SIGDIAL
SIG:
SIGDIAL
Publisher:
Association for Computational Linguistics
Note:
Pages:
331–341
Language:
URL:
https://aclanthology.org/W17-5539
DOI:
10.18653/v1/W17-5539
Bibkey:
Cite (ACL):
Ramesh Manuvinakurike, David DeVault, and Kallirroi Georgila. 2017. Using Reinforcement Learning to Model Incrementality in a Fast-Paced Dialogue Game. In Proceedings of the 18th Annual SIGdial Meeting on Discourse and Dialogue, pages 331–341, Saarbrücken, Germany. Association for Computational Linguistics.
Cite (Informal):
Using Reinforcement Learning to Model Incrementality in a Fast-Paced Dialogue Game (Manuvinakurike et al., SIGDIAL 2017)
Copy Citation:
PDF:
https://aclanthology.org/W17-5539.pdf