Reinforced Video Captioning with Entailment Rewards

Ramakanth Pasunuru, Mohit Bansal


Abstract
Sequence-to-sequence models have shown promising improvements on the temporal task of video captioning, but they optimize word-level cross-entropy loss during training. First, using policy gradient and mixed-loss methods for reinforcement learning, we directly optimize sentence-level task-based metrics (as rewards), achieving significant improvements over the baseline, based on both automatic metrics and human evaluation on multiple datasets. Next, we propose a novel entailment-enhanced reward (CIDEnt) that corrects phrase-matching based metrics (such as CIDEr) to only allow for logically-implied partial matches and avoid contradictions, achieving further significant improvements over the CIDEr-reward model. Overall, our CIDEnt-reward model achieves the new state-of-the-art on the MSR-VTT dataset.
Anthology ID:
D17-1103
Volume:
Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing
Month:
September
Year:
2017
Address:
Copenhagen, Denmark
Editors:
Martha Palmer, Rebecca Hwa, Sebastian Riedel
Venue:
EMNLP
SIG:
SIGDAT
Publisher:
Association for Computational Linguistics
Note:
Pages:
979–985
Language:
URL:
https://aclanthology.org/D17-1103
DOI:
10.18653/v1/D17-1103
Bibkey:
Cite (ACL):
Ramakanth Pasunuru and Mohit Bansal. 2017. Reinforced Video Captioning with Entailment Rewards. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 979–985, Copenhagen, Denmark. Association for Computational Linguistics.
Cite (Informal):
Reinforced Video Captioning with Entailment Rewards (Pasunuru & Bansal, EMNLP 2017)
Copy Citation:
PDF:
https://aclanthology.org/D17-1103.pdf
Attachment:
 D17-1103.Attachment.pdf
Data
MSR-VTT