Difference between revisions of "POS Tagging (State of the art)"

From ACL Wiki
Jump to: navigation, search
Line 1: Line 1:
 
* '''Performance measure:''' per token accuracy. (The convention is for this to be measured on all tokens, including punctuation tokens and other unambiguous tokens.)
 
* '''Performance measure:''' per token accuracy. (The convention is for this to be measured on all tokens, including punctuation tokens and other unambiguous tokens.)
* '''Training data:''' sections 0-18 of Wall Street Journal corpus
 
* '''Testing data:''' sections 22-24 of Wall Street Journal corpus
 
  
 +
==Test collections==
 +
* '''English'''
 +
** '''Penn Treebank''' ''Wall Street Journal'' (WSJ). The splits of data for this data set were not standardized early on (unlike for parsing) and early work uses various data splits defined by counts of tokens or by sections.  Most work from 2002 on adopts the following data splits, introduced by Collins (2002):
 +
*** '''Training data:''' sections 0-18
 +
*** '''Development test data:''' sections 19-21
 +
*** '''Testing data:''' sections 22-24
  
== Table of results ==
 
  
 +
== Tables of results ==
 +
 +
===WSJ===
  
 
{| border="1" cellpadding="5" cellspacing="1" width="100%"
 
{| border="1" cellpadding="5" cellspacing="1" width="100%"
Line 13: Line 19:
 
! Main publications
 
! Main publications
 
! Software
 
! Software
! Results
+
! All tokens
 +
! Unknown words
 +
|-
 +
| Averaged Perceptron
 +
| Averaged Perception discriminative sequence model
 +
| Collins (2002)
 +
| Not available
 +
| 97.11%
 +
| Not available
 
|-
 
|-
 
| SVMTool
 
| SVMTool
Line 20: Line 34:
 
| [http://www.lsi.upc.es/~nlp/SVMTool/ SVMTool]
 
| [http://www.lsi.upc.es/~nlp/SVMTool/ SVMTool]
 
| 97.16%
 
| 97.16%
 +
| 89.01%
 
|-
 
|-
| Stanford Tagger
+
| Stanford Tagger 1.0
| learning with cyclic dependency network
+
| maximum entropy cyclic dependency network
 
| Toutanova et al. (2003)
 
| Toutanova et al. (2003)
 
| [http://nlp.stanford.edu/software/tagger.shtml Stanford Tagger]
 
| [http://nlp.stanford.edu/software/tagger.shtml Stanford Tagger]
Line 43: Line 58:
 
== References ==
 
== References ==
  
* Giménez, J., and Márquez, L. (2004). [http://www.lsi.upc.es/~nlp/SVMTool/lrec2004-gm.pdf SVMTool: A general POS tagger generator based on Support Vector Machines]. ''Proceedings of the 4th International Conference on Language Resources and Evaluation (LREC'04)''. Lisbon, Portugal.  
+
* Collins, Michael. 2002. [http://people.csail.mit.edu/mcollins/papers/tagperc.pdf Discriminative Training Methods for Hidden Markov Models: Theory and Experiments with Perceptron Algorithms]. ''EMNLP 2002''.
 +
 
 +
* Giménez, J., and Márquez, L. 2004. [http://www.lsi.upc.es/~nlp/SVMTool/lrec2004-gm.pdf SVMTool: A general POS tagger generator based on Support Vector Machines]. ''Proceedings of the 4th International Conference on Language Resources and Evaluation (LREC'04)''. Lisbon, Portugal.  
  
* Shen, L., Satta, G., and  Joshi, A. (2007). [http://acl.ldc.upenn.edu/P/P07/P07-1096.pdf Guided learning for bidirectional sequence classification]. ''Proceedings of the 45th Annual Meeting of the Association of Computational Linguistics (ACL 2007)'', pages 760-767.
+
* Shen, L., Satta, G., and  Joshi, A. 2007. [http://acl.ldc.upenn.edu/P/P07/P07-1096.pdf Guided learning for bidirectional sequence classification]. ''Proceedings of the 45th Annual Meeting of the Association of Computational Linguistics (ACL 2007)'', pages 760-767.
  
* Toutanova, K., Klein, D., Manning, C.D., Yoram Singer, Y. (2003) [http://nlp.stanford.edu/kristina/papers/tagging.pdf Feature-rich part-of-speech tagging with a cyclic dependency network]. ''Proceedings of HLT-NAACL 2003'', pages 252-259.
+
* Toutanova, K., Klein, D., Manning, C.D., Yoram Singer, Y. 2003. [http://nlp.stanford.edu/kristina/papers/tagging.pdf Feature-rich part-of-speech tagging with a cyclic dependency network]. ''Proceedings of HLT-NAACL 2003'', pages 252-259.
  
* Yoshimasa Tsuruoka, Yuka Tateishi, Jin-Dong Kim, Tomoko Ohta, John McNaught, Sophia Ananiadou, and Jun'ichi Tsujii, "[http://www-tsujii.is.s.u-tokyo.ac.jp/~tsuruoka/papers/pci05.pdf Developing a Robust Part-of-Speech Tagger for Biomedical Text, Advances in Informatics]" - ''10th Panhellenic Conference on Informatics'', '''LNCS 3746''', pp. 382-392, 2005  
+
* Tsuruoka, Yoshimasa, Yuka Tateishi, Jin-Dong Kim, Tomoko Ohta, John McNaught, Sophia Ananiadou, and Jun'ichi Tsujii. 2005. "[http://www-tsujii.is.s.u-tokyo.ac.jp/~tsuruoka/papers/pci05.pdf Developing a Robust Part-of-Speech Tagger for Biomedical Text, Advances in Informatics]" - ''10th Panhellenic Conference on Informatics'', '''LNCS 3746''', pp. 382-392, 2005  
  
* Yoshimasa Tsuruoka and Jun'ichi Tsujii, "[http://www-tsujii.is.s.u-tokyo.ac.jp/~tsuruoka/papers/emnlp05bidir.pdf Bidirectional Inference with the Easiest-First Strategy for Tagging Sequence Data]", ''Proceedings of HLT/EMNLP 2005'', pp. 467-474.
+
* Tsuruoka, Yoshimasa and Jun'ichi Tsujii. 2005. "[http://www-tsujii.is.s.u-tokyo.ac.jp/~tsuruoka/papers/emnlp05bidir.pdf Bidirectional Inference with the Easiest-First Strategy for Tagging Sequence Data]", ''Proceedings of HLT/EMNLP 2005'', pp. 467-474.
  
 
== See also ==
 
== See also ==

Revision as of 21:34, 1 January 2010

  • Performance measure: per token accuracy. (The convention is for this to be measured on all tokens, including punctuation tokens and other unambiguous tokens.)

Test collections

  • English
    • Penn Treebank Wall Street Journal (WSJ). The splits of data for this data set were not standardized early on (unlike for parsing) and early work uses various data splits defined by counts of tokens or by sections. Most work from 2002 on adopts the following data splits, introduced by Collins (2002):
      • Training data: sections 0-18
      • Development test data: sections 19-21
      • Testing data: sections 22-24


Tables of results

WSJ

System name Short description Main publications Software All tokens Unknown words
Averaged Perceptron Averaged Perception discriminative sequence model Collins (2002) Not available 97.11% Not available
SVMTool SVM-based tagger and tagger generator Giménez and Márquez (2004) SVMTool 97.16% 89.01%
Stanford Tagger 1.0 maximum entropy cyclic dependency network Toutanova et al. (2003) Stanford Tagger 97.24%
LTAG-spinal bidirectional perceptron learning Shen et al. (2007) LTAG-spinal 97.33%
GENiA Tagger  ? Tsuruoka, et al (2005) GENiA 96.94% on WSJ, 98.26% on biomed.

References

See also