View source for MUC-7 (State of the art)

* '''Performance measure:''' F = 2 * Precision * Recall / (Recall + Precision)
* '''Precision:''' percentage of named entities found by the algorithm that are correct
* '''Recall:''' percentage of named entities defined in the corpus that were found by the program
* Exact calculation of precision and recall is explained in the [http://www.itl.nist.gov/iad/894.02/related_projects/muc/muc_sw/muc_sw_manual.html MUC scoring software]


* '''Training data:''' Training section of MUC-7 dataset
* '''Dryrun data:''' Dryrun section of MUC-7 dataset
* '''Testing data:''' Formal section of MUC-7 dataset


== Table of results ==


{| border="1" cellpadding="5" cellspacing="1" width="100%"
|-
! System name
! Short description
! System type (1)
! Main publications
! Software
! Results
|-
| Annotator
| Human annotator
| -
| [http://www.itl.nist.gov/iad/894.02/related_projects/muc/proceedings/muc_7_toc.html MUC-7 proceedings]
| -
| 97.60%
|-
| LTG
| Best MUC-7 participant
| H
| Mikheev, Grover and Moens (1998)
| -
| 93.39%
|-
| Balie
| Unsupervised approach: no prior training
| U
| Nadeau, Turney and Matwin (2006)
| [http://balie.sourceforge.net sourceforge.net]
| 77.71% (2)
|-
| Baseline
| Vocabulary transfer from training to testing
| S
| Whitelaw and Patrick (2003)
| -
| 58.89% (2)
|-
|}

* (1) '''System type''': R = hand-crafted rules, S = supervised learning, U = unsupervised learning, H = hybrid 
* (2) Calculated on Enamex types only.


== References ==

Mikheev, A., Grover, C. and Moens, M. (1998). [http://www-nlpir.nist.gov/related_projects/muc/proceedings/muc_7_proceedings/ltg_muc7.pdf Description of the LTG system used for MUC-7]. ''Proceedings of the Seventh Message Understanding Conference (MUC-7)''. Fairfax, Virginia.

Nadeau, D., Turney, P. D. and Matwin, S. (2006) [http://iit-iti.nrc-cnrc.gc.ca/publications/nrc-48727_e.html Unsupervised Named-Entity Recognition: Generating Gazetteers and Resolving Ambiguity]. ''Proceedings 19th Canadian Conference on Artificial Intelligence''. Québec, Canada.

Whitelaw, C. and Patrick, J. (2003) [http://www.springerlink.com/content/ju66c6a2734fl20u/ Evaluating Corpora for Named Entity Recognition Using Character-Level Features]. ''Proceeding of the 16th Australian Conference on AI''. Perth, Australia. 

== See also ==

* [[Named Entity Recognition (State of the art)|Named Entity Recognition]]
* [[State of the art]]

[[Category:State of the art]]