Sentence-State LSTM for Text Representation

Yue Zhang, Qi Liu, Linfeng Song


Abstract
Bi-directional LSTMs are a powerful tool for text representation. On the other hand, they have been shown to suffer various limitations due to their sequential nature. We investigate an alternative LSTM structure for encoding text, which consists of a parallel state for each word. Recurrent steps are used to perform local and global information exchange between words simultaneously, rather than incremental reading of a sequence of words. Results on various classification and sequence labelling benchmarks show that the proposed model has strong representation power, giving highly competitive performances compared to stacked BiLSTM models with similar parameter numbers.
Anthology ID:
P18-1030
Volume:
Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
Month:
July
Year:
2018
Address:
Melbourne, Australia
Editors:
Iryna Gurevych, Yusuke Miyao
Venue:
ACL
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
317–327
Language:
URL:
https://aclanthology.org/P18-1030
DOI:
10.18653/v1/P18-1030
Bibkey:
Cite (ACL):
Yue Zhang, Qi Liu, and Linfeng Song. 2018. Sentence-State LSTM for Text Representation. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 317–327, Melbourne, Australia. Association for Computational Linguistics.
Cite (Informal):
Sentence-State LSTM for Text Representation (Zhang et al., ACL 2018)
Copy Citation:
PDF:
https://aclanthology.org/P18-1030.pdf
Poster:
 P18-1030.Poster.pdf
Code
 additional community code
Data
CoNLLCoNLL 2003IMDb Movie ReviewsMRPenn Treebank