WordSimilarity-353 Test Collection (State of the art)
The printable version is no longer supported and may have rendering errors. Please update your browser bookmarks and please use the default browser print function instead.
- WordSimilarity-353 Test Collection
- contains two sets of English word pairs along with human-assigned similarity judgements
- first set (set1) contains 153 word pairs along with their similarity scores assigned by 13 subjects
- second set (set2) contains 200 word pairs with similarity assessed by 16 subjects
- WordSimilarity-353 dataset is available here
- performance is measured by Spearman's rank correlation coefficient
- introduced by Finkelstein et al. (2002)
- subsequently used by many other researchers
0.75 Spearman rank correlation
http://www.cs.technion.ac.il/~gabr/papers/ijcai-2007-sim.pdf
0.65 Spearman rank correlation http://nlp.stanford.edu/~lmthang/data/papers/conll13_morpho.pdf
Table of results
- Listed in order of increasing Spearman's rho.
| Algorithm | Reference | Spearman's rho |
|---|---|---|
| HSMN+csmRNN | Luong et al. (2013) | 0.646 |
References
Luong, Minh-Thang, Richard Socher, and Christopher D. Manning. (2013) Better word representations with recursive neural networks for morphology. CoNLL-2013: 104.