Difference between revisions of "WordSimilarity-353 Test Collection (State of the art)"

From ACL Wiki
Jump to navigation Jump to search
(Created page with " WordSimilarity-353 Test Collection http://www.cs.technion.ac.il/~gabr/resources/data/wordsim353/ 0.75 Spearman rank correlation http://www.cs.technion.ac.il/~gabr/papers/ijc...")
 
(18 intermediate revisions by 2 users not shown)
Line 1: Line 1:
  
−
WordSimilarity-353 Test Collection
+
* [http://www.cs.technion.ac.il/~gabr/resources/data/wordsim353/ WordSimilarity-353 Test Collection]
−
http://www.cs.technion.ac.il/~gabr/resources/data/wordsim353/
+
* contains two sets of English word pairs along with human-assigned similarity judgements
 +
* first set (set1) contains 153 word pairs along with their similarity scores assigned by 13 subjects
 +
* second set (set2) contains 200 word pairs with similarity assessed by 16 subjects
 +
* WordSimilarity-353 dataset is available [http://www.cs.technion.ac.il/~gabr/resources/data/wordsim353/ here]
 +
* performance is measured by [http://en.wikipedia.org/wiki/Spearman_rank_correlation Spearman's rank correlation coefficient]
 +
* introduced by [http://www.cs.technion.ac.il/~gabr/papers/tois_context.pdf Finkelstein et al. (2002)]
 +
* subsequently used by many other researchers
 +
* see also: [[Similarity (State of the art)]]
  
−
0.75 Spearman rank correlation
 
−
http://www.cs.technion.ac.il/~gabr/papers/ijcai-2007-sim.pdf
 
  
−
0.65 Spearman rank correlation
+
== Table of results ==
−
http://nlp.stanford.edu/~lmthang/data/papers/conll13_morpho.pdf
+
 
 +
* '''Listed in order of increasing [http://en.wikipedia.org/wiki/Spearman_rank_correlation Spearman's rho].'''
 +
 
 +
{| border="1" cellpadding="5" cellspacing="1"
 +
|-
 +
! Algorithm
 +
! Reference
 +
! Type
 +
! Spearman's rho
 +
|-
 +
| WikiRelate
 +
| Strube and Ponzetto (2006)
 +
| Wikipedia
 +
| 0.48
 +
|-
 +
| C&W
 +
| Collobert and Weston (2008)
 +
| Corpus-based
 +
| 0.498
 +
|-
 +
| simVB+simWN
 +
| Finkelstein et al. (2002)
 +
| Hybrid
 +
| 0.55
 +
|-
 +
| HSMN+csmRNN
 +
| Luong et al. (2013)
 +
| Corpus-based
 +
| 0.65
 +
|-
 +
| multi-prototype
 +
| Huang et al. (2012)
 +
| Corpus-based
 +
| 0.713
 +
|-
 +
| ESA-Wikipedia
 +
| Gabrilovich and Markovitch (2007)
 +
| Hybrid
 +
| 0.75
 +
|-
 +
| TSA
 +
| Radinsky et al. (2011)
 +
| Hybrid
 +
| 0.800
 +
|-
 +
| CLEAR
 +
| Halawi et al. (2012)
 +
| Corpus-based
 +
| 0.810
 +
|}
 +
 
 +
== References ==
 +
 
 +
Finkelstein, Lev, Evgeniy Gabrilovich, Yossi Matias, Ehud Rivlin, Zach Solan, Gadi Wolfman, and Eytan Ruppin. (2002) [http://www.cs.technion.ac.il/~gabr/papers/tois_context.pdf Placing Search in Context: The Concept Revisited]. ACM Transactions on Information Systems, 20(1):116-131.
 +
 
 +
Gabrilovich, Evgeniy, and Shaul Markovitch. (2007). [http://www.cs.technion.ac.il/~gabr/papers/ijcai-2007-sim.pdf Computing Semantic Relatedness Using Wikipedia-based Explicit Semantic Analysis]. In IJCAI, vol. 7, pp. 1606-1611.
 +
 
 +
Luong, Minh-Thang, Richard Socher, and Christopher D. Manning. (2013). [http://nlp.stanford.edu/~lmthang/data/papers/conll13_morpho.pdf Better word representations with recursive neural networks for morphology]. CoNLL-2013: 104.
 +
 
 +
Strube, Michael and Simone Paolo Ponzetto. (2006). [http://www.aaai.org/Papers/AAAI/2006/AAAI06-223.pdf WikiRelate! Computing Semantic Relatedness Using Wikipedia]. Proceedings of The 21st National Conference on Artificial Intelligence (AAAI), Boston, MA.
 +
 
 +
 
 +
[[Category:State of the art]]

Revision as of 14:30, 11 November 2013


Table of results

Algorithm Reference Type Spearman's rho
WikiRelate Strube and Ponzetto (2006) Wikipedia 0.48
C&W Collobert and Weston (2008) Corpus-based 0.498
simVB+simWN Finkelstein et al. (2002) Hybrid 0.55
HSMN+csmRNN Luong et al. (2013) Corpus-based 0.65
multi-prototype Huang et al. (2012) Corpus-based 0.713
ESA-Wikipedia Gabrilovich and Markovitch (2007) Hybrid 0.75
TSA Radinsky et al. (2011) Hybrid 0.800
CLEAR Halawi et al. (2012) Corpus-based 0.810

References

Finkelstein, Lev, Evgeniy Gabrilovich, Yossi Matias, Ehud Rivlin, Zach Solan, Gadi Wolfman, and Eytan Ruppin. (2002) Placing Search in Context: The Concept Revisited. ACM Transactions on Information Systems, 20(1):116-131.

Gabrilovich, Evgeniy, and Shaul Markovitch. (2007). Computing Semantic Relatedness Using Wikipedia-based Explicit Semantic Analysis. In IJCAI, vol. 7, pp. 1606-1611.

Luong, Minh-Thang, Richard Socher, and Christopher D. Manning. (2013). Better word representations with recursive neural networks for morphology. CoNLL-2013: 104.

Strube, Michael and Simone Paolo Ponzetto. (2006). WikiRelate! Computing Semantic Relatedness Using Wikipedia. Proceedings of The 21st National Conference on Artificial Intelligence (AAAI), Boston, MA.

Retrieved from "https://aclweb.org/aclwiki/index.php?title=WordSimilarity-353_Test_Collection_(State_of_the_art)&oldid=10425"