Disconnected Recurrent Neural Networks for Text Categorization

Baoxin Wang


Abstract
Recurrent neural network (RNN) has achieved remarkable performance in text categorization. RNN can model the entire sequence and capture long-term dependencies, but it does not do well in extracting key patterns. In contrast, convolutional neural network (CNN) is good at extracting local and position-invariant features. In this paper, we present a novel model named disconnected recurrent neural network (DRNN), which incorporates position-invariance into RNN. By limiting the distance of information flow in RNN, the hidden state at each time step is restricted to represent words near the current position. The proposed model makes great improvements over RNN and CNN models and achieves the best performance on several benchmark datasets for text categorization.
Anthology ID:
P18-1215
Volume:
Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
Month:
July
Year:
2018
Address:
Melbourne, Australia
Editors:
Iryna Gurevych, Yusuke Miyao
Venue:
ACL
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
2311–2320
Language:
URL:
https://aclanthology.org/P18-1215
DOI:
10.18653/v1/P18-1215
Bibkey:
Cite (ACL):
Baoxin Wang. 2018. Disconnected Recurrent Neural Networks for Text Categorization. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 2311–2320, Melbourne, Australia. Association for Computational Linguistics.
Cite (Informal):
Disconnected Recurrent Neural Networks for Text Categorization (Wang, ACL 2018)
Copy Citation:
PDF:
https://aclanthology.org/P18-1215.pdf
Poster:
 P18-1215.Poster.pdf
Data
AG NewsDBpediaYahoo! AnswersYelp