Unsupervised Sentence Compression using Denoising Auto-Encoders

Thibault Févry, Jason Phang


Abstract
In sentence compression, the task of shortening sentences while retaining the original meaning, models tend to be trained on large corpora containing pairs of verbose and compressed sentences. To remove the need for paired corpora, we emulate a summarization task and add noise to extend sentences and train a denoising auto-encoder to recover the original, constructing an end-to-end training regime without the need for any examples of compressed sentences. We conduct a human evaluation of our model on a standard text summarization dataset and show that it performs comparably to a supervised baseline based on grammatical correctness and retention of meaning. Despite being exposed to no target data, our unsupervised models learn to generate imperfect but reasonably readable sentence summaries. Although we underperform supervised models based on ROUGE scores, our models are competitive with a supervised baseline based on human evaluation for grammatical correctness and retention of meaning.
Anthology ID:
K18-1040
Volume:
Proceedings of the 22nd Conference on Computational Natural Language Learning
Month:
October
Year:
2018
Address:
Brussels, Belgium
Editors:
Anna Korhonen, Ivan Titov
Venue:
CoNLL
SIG:
SIGNLL
Publisher:
Association for Computational Linguistics
Note:
Pages:
413–422
Language:
URL:
https://aclanthology.org/K18-1040
DOI:
10.18653/v1/K18-1040
Bibkey:
Cite (ACL):
Thibault Févry and Jason Phang. 2018. Unsupervised Sentence Compression using Denoising Auto-Encoders. In Proceedings of the 22nd Conference on Computational Natural Language Learning, pages 413–422, Brussels, Belgium. Association for Computational Linguistics.
Cite (Informal):
Unsupervised Sentence Compression using Denoising Auto-Encoders (Févry & Phang, CoNLL 2018)
Copy Citation:
PDF:
https://aclanthology.org/K18-1040.pdf
Code
 zphang/usc_dae