Semi-supervised Multitask Learning for Sequence Labeling

Marek Rei


Abstract
We propose a sequence labeling framework with a secondary training objective, learning to predict surrounding words for every word in the dataset. This language modeling objective incentivises the system to learn general-purpose patterns of semantic and syntactic composition, which are also useful for improving accuracy on different sequence labeling tasks. The architecture was evaluated on a range of datasets, covering the tasks of error detection in learner texts, named entity recognition, chunking and POS-tagging. The novel language modeling objective provided consistent performance improvements on every benchmark, without requiring any additional annotated or unannotated data.
Anthology ID:
P17-1194
Volume:
Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
Month:
July
Year:
2017
Address:
Vancouver, Canada
Editors:
Regina Barzilay, Min-Yen Kan
Venue:
ACL
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
2121–2130
Language:
URL:
https://aclanthology.org/P17-1194
DOI:
10.18653/v1/P17-1194
Bibkey:
Cite (ACL):
Marek Rei. 2017. Semi-supervised Multitask Learning for Sequence Labeling. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 2121–2130, Vancouver, Canada. Association for Computational Linguistics.
Cite (Informal):
Semi-supervised Multitask Learning for Sequence Labeling (Rei, ACL 2017)
Copy Citation:
PDF:
https://aclanthology.org/P17-1194.pdf
Poster:
 P17-1194.Poster.pdf
Code
 marekrei/sequence-labeler +  additional community code
Data
CoNLLCoNLL 2003FCEPenn Treebank