Generating Sentences by Editing Prototypes

Kelvin Guu, Tatsunori B. Hashimoto, Yonatan Oren, Percy Liang


Abstract
We propose a new generative language model for sentences that first samples a prototype sentence from the training corpus and then edits it into a new sentence. Compared to traditional language models that generate from scratch either left-to-right or by first sampling a latent sentence vector, our prototype-then-edit model improves perplexity on language modeling and generates higher quality outputs according to human evaluation. Furthermore, the model gives rise to a latent edit vector that captures interpretable semantics such as sentence similarity and sentence-level analogies.
Anthology ID:
Q18-1031
Volume:
Transactions of the Association for Computational Linguistics, Volume 6
Month:
Year:
2018
Address:
Cambridge, MA
Editors:
Lillian Lee, Mark Johnson, Kristina Toutanova, Brian Roark
Venue:
TACL
SIG:
Publisher:
MIT Press
Note:
Pages:
437–450
Language:
URL:
https://aclanthology.org/Q18-1031
DOI:
10.1162/tacl_a_00030
Bibkey:
Cite (ACL):
Kelvin Guu, Tatsunori B. Hashimoto, Yonatan Oren, and Percy Liang. 2018. Generating Sentences by Editing Prototypes. Transactions of the Association for Computational Linguistics, 6:437–450.
Cite (Informal):
Generating Sentences by Editing Prototypes (Guu et al., TACL 2018)
Copy Citation:
PDF:
https://aclanthology.org/Q18-1031.pdf
Video:
 https://aclanthology.org/Q18-1031.mp4
Code
 kelvinguu/neural-editor +  additional community code
Data
Billion Word BenchmarkOne Billion Word Benchmark