Monte Carlo Syntax Marginals for Exploring and Using Dependency Parses

Katherine Keith, Su Lin Blodgett, Brendan O’Connor


Abstract
Dependency parsing research, which has made significant gains in recent years, typically focuses on improving the accuracy of single-tree predictions. However, ambiguity is inherent to natural language syntax, and communicating such ambiguity is important for error analysis and better-informed downstream applications. In this work, we propose a transition sampling algorithm to sample from the full joint distribution of parse trees defined by a transition-based parsing model, and demonstrate the use of the samples in probabilistic dependency analysis. First, we define the new task of dependency path prediction, inferring syntactic substructures over part of a sentence, and provide the first analysis of performance on this task. Second, we demonstrate the usefulness of our Monte Carlo syntax marginal method for parser error analysis and calibration. Finally, we use this method to propagate parse uncertainty to two downstream information extraction applications: identifying persons killed by police and semantic role assignment.
Anthology ID:
N18-1084
Volume:
Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers)
Month:
June
Year:
2018
Address:
New Orleans, Louisiana
Editors:
Marilyn Walker, Heng Ji, Amanda Stent
Venue:
NAACL
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
917–928
Language:
URL:
https://aclanthology.org/N18-1084
DOI:
10.18653/v1/N18-1084
Bibkey:
Cite (ACL):
Katherine Keith, Su Lin Blodgett, and Brendan O’Connor. 2018. Monte Carlo Syntax Marginals for Exploring and Using Dependency Parses. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers), pages 917–928, New Orleans, Louisiana. Association for Computational Linguistics.
Cite (Informal):
Monte Carlo Syntax Marginals for Exploring and Using Dependency Parses (Keith et al., NAACL 2018)
Copy Citation:
PDF:
https://aclanthology.org/N18-1084.pdf
Code
 slanglab/transition_sampler
Data
OntoNotes 5.0Universal Dependencies