Exploration of Noise Strategies in Semi-supervised Named Entity Classification

Pooja Lakshmi Narayan, Ajay Nagesh, Mihai Surdeanu


Abstract
Noise is inherent in real world datasets and modeling noise is critical during training as it is effective in regularization. Recently, novel semi-supervised deep learning techniques have demonstrated tremendous potential when learning with very limited labeled training data in image processing tasks. A critical aspect of these semi-supervised learning techniques is augmenting the input or the network with noise to be able to learn robust models. While modeling noise is relatively straightforward in continuous domains such as image classification, it is not immediately apparent how noise can be modeled in discrete domains such as language. Our work aims to address this gap by exploring different noise strategies for the semi-supervised named entity classification task, including statistical methods such as adding Gaussian noise to input embeddings, and linguistically-inspired ones such as dropping words and replacing words with their synonyms. We compare their performance on two benchmark datasets (OntoNotes and CoNLL) for named entity classification. Our results indicate that noise strategies that are linguistically informed perform at least as well as statistical approaches, while being simpler and requiring minimal tuning.
Anthology ID:
S19-1020
Volume:
Proceedings of the Eighth Joint Conference on Lexical and Computational Semantics (*SEM 2019)
Month:
June
Year:
2019
Address:
Minneapolis, Minnesota
Editors:
Rada Mihalcea, Ekaterina Shutova, Lun-Wei Ku, Kilian Evang, Soujanya Poria
Venue:
*SEM
SIGs:
SIGLEX | SIGSEM
Publisher:
Association for Computational Linguistics
Note:
Pages:
186–191
Language:
URL:
https://aclanthology.org/S19-1020
DOI:
10.18653/v1/S19-1020
Bibkey:
Cite (ACL):
Pooja Lakshmi Narayan, Ajay Nagesh, and Mihai Surdeanu. 2019. Exploration of Noise Strategies in Semi-supervised Named Entity Classification. In Proceedings of the Eighth Joint Conference on Lexical and Computational Semantics (*SEM 2019), pages 186–191, Minneapolis, Minnesota. Association for Computational Linguistics.
Cite (Informal):
Exploration of Noise Strategies in Semi-supervised Named Entity Classification (Lakshmi Narayan et al., *SEM 2019)
Copy Citation:
PDF:
https://aclanthology.org/S19-1020.pdf
Data
OntoNotes 5.0