Difference between revisions of "WordSimilarity-353 Test Collection (State of the art)"

From ACL Wiki
Jump to: navigation, search
(References)
(Table of results)
Line 18: Line 18:
 
|-
 
|-
 
! Algorithm
 
! Algorithm
! Reference
+
! Reference for algorithm
 +
! Reference for reported results
 
! Type
 
! Type
 
! Spearman's rho
 
! Spearman's rho
 +
! Pearson's r
 +
|-
 +
| L&C
 +
| Leacock and Chodorow (1998)
 +
| Hassan and Mihalcea (2011)
 +
| Knowledge-based
 +
| 0.302
 +
| 0.356
 +
|-
 +
| WNE
 +
| Jarmasz (2003)
 +
| Hassan and Mihalcea (2011)
 +
| Knowledge-based
 +
| 0.305
 +
| 0.271
 +
|-
 +
| J&C
 +
| Jiang and Conrath 1997
 +
| Hassan and Mihalcea (2011)
 +
| Knowledge-based
 +
| 0.318
 +
| 0.354
 +
|-
 +
| L&C
 +
| Leacock and Chodorow (1998)
 +
| Hassan and Mihalcea (2011)
 +
| Knowledge-based
 +
| 0.348
 +
| 0.341
 +
|-
 +
| H&S
 +
| Hirst and St-Onge (1998)
 +
| Hassan and Mihalcea (2011)
 +
| Knowledge-based
 +
| 0.302
 +
| 0.356
 +
|-
 +
| Lin
 +
| Lin (1998)
 +
| Hassan and Mihalcea (2011)
 +
| Corpus-based
 +
| 0.348
 +
| 0.357
 +
|-
 +
| Resnik
 +
| Resnik (1995)
 +
| Hassan and Mihalcea (2011)
 +
| Knowledge-based
 +
| 0.353
 +
| 0.365
 +
|-
 +
| ROGET
 +
| Jarmasz (2003)
 +
| Hassan and Mihalcea (2011)
 +
| Knowledge-based
 +
| 0.415
 +
| 0.536
 +
|-
 +
| C&W
 +
| Collobert and Weston (2008)
 +
| Collobert and Weston (2008)
 +
| Corpus-based
 +
| 0.5
 +
| N/A
 
|-
 
|-
 
| WikiRelate
 
| WikiRelate
 
| Strube and Ponzetto (2006)
 
| Strube and Ponzetto (2006)
| Wikipedia
+
| Strube and Ponzetto (2006)
 +
| Corpus-based
 +
| N/A
 
| 0.48
 
| 0.48
 
|-
 
|-
| C&W
+
| LSA
| Collobert and Weston (2008)
+
| Landauer et al. (1997)
 +
| Hassan and Mihalcea (2011)
 +
| Corpus-based
 +
| 0.581
 +
| 0.492
 +
|-
 +
| LSA
 +
| Landauer et al. (1997)
 +
| Hassan and Mihalcea (2011)
 
| Corpus-based
 
| Corpus-based
| 0.50
+
| 0.581
 +
| 0.563
 
|-
 
|-
 
| simVB+simWN
 
| simVB+simWN
 +
| Finkelstein et al. (2002)
 
| Finkelstein et al. (2002)
 
| Finkelstein et al. (2002)
 
| Hybrid
 
| Hybrid
 +
| N/A
 
| 0.55
 
| 0.55
 +
|-
 +
| SSA
 +
| Hassan and Mihalcea (2011)
 +
| Hassan and Mihalcea (2011)
 +
| Knowledge-based
 +
| 0.622
 +
| 0.629
 
|-
 
|-
 
| HSMN+csmRNN
 
| HSMN+csmRNN
 +
| Luong et al. (2013)
 
| Luong et al. (2013)
 
| Luong et al. (2013)
 
| Corpus-based
 
| Corpus-based
 
| 0.65
 
| 0.65
 +
| N/A
 
|-
 
|-
| multi-prototype
+
| Multi-prototype
 +
| Huang et al. (2012)
 
| Huang et al. (2012)
 
| Huang et al. (2012)
 
| Corpus-based
 
| Corpus-based
 
| 0.71
 
| 0.71
 +
| N/A
 
|-
 
|-
| ESA-Wikipedia
+
| Multi-lingual SSA
 +
| Hassan et al. (2011)
 +
| Hassan et al. (2011)
 +
| Corpus-based
 +
| 0.713
 +
| 0.674
 +
|-
 +
| ESA
 
| Gabrilovich and Markovitch (2007)
 
| Gabrilovich and Markovitch (2007)
| Hybrid
+
| Gabrilovich and Markovitch (2007)
| 0.75
+
| Corpus-based
 +
| 0.748
 +
| 0.503
 
|-
 
|-
 
| TSA
 
| TSA
 +
| Radinsky et al. (2011)
 
| Radinsky et al. (2011)
 
| Radinsky et al. (2011)
 
| Hybrid
 
| Hybrid
 
| 0.80
 
| 0.80
 +
| N/A
 
|-
 
|-
 
| CLEAR
 
| CLEAR
 +
| Halawi et al. (2012)
 
| Halawi et al. (2012)
 
| Halawi et al. (2012)
 
| Corpus-based
 
| Corpus-based
 
| 0.81
 
| 0.81
|}
+
| N/A
 +
|-
 +
}
  
 
== References ==
 
== References ==

Revision as of 19:11, 7 January 2014


Table of results

}

References

Finkelstein, Lev, Evgeniy Gabrilovich, Yossi Matias, Ehud Rivlin, Zach Solan, Gadi Wolfman, and Eytan Ruppin. (2002) Placing Search in Context: The Concept Revisited. ACM Transactions on Information Systems, 20(1):116-131.

Gabrilovich, Evgeniy, and Shaul Markovitch. (2007). Computing Semantic Relatedness Using Wikipedia-based Explicit Semantic Analysis. In IJCAI, vol. 7, pp. 1606-1611.

Halawi, Guy, Gideon Dror, Evgeniy Gabrilovich, and Yehuda Koren. (2012). Large-scale learning of word relatedness with constraints. In Proceedings of the 18th ACM SIGKDD international conference on Knowledge discovery and data mining, pp. 1406-1414. ACM.

Luong, Minh-Thang, Richard Socher, and Christopher D. Manning. (2013). Better word representations with recursive neural networks for morphology. CoNLL-2013: 104.

Radinsky, Kira, Eugene Agichtein, Evgeniy Gabrilovich, and Shaul Markovitch. (2011). A word at a time: computing word relatedness using temporal semantic analysis. In Proceedings of the 20th international conference on World wide web, pp. 337-346. ACM.

Strube, Michael and Simone Paolo Ponzetto. (2006). WikiRelate! Computing Semantic Relatedness Using Wikipedia. Proceedings of The 21st National Conference on Artificial Intelligence (AAAI), Boston, MA.

Algorithm Reference for algorithm Reference for reported results Type Spearman's rho Pearson's r
L&C Leacock and Chodorow (1998) Hassan and Mihalcea (2011) Knowledge-based 0.302 0.356
WNE Jarmasz (2003) Hassan and Mihalcea (2011) Knowledge-based 0.305 0.271
J&C Jiang and Conrath 1997 Hassan and Mihalcea (2011) Knowledge-based 0.318 0.354
L&C Leacock and Chodorow (1998) Hassan and Mihalcea (2011) Knowledge-based 0.348 0.341
H&S Hirst and St-Onge (1998) Hassan and Mihalcea (2011) Knowledge-based 0.302 0.356
Lin Lin (1998) Hassan and Mihalcea (2011) Corpus-based 0.348 0.357
Resnik Resnik (1995) Hassan and Mihalcea (2011) Knowledge-based 0.353 0.365
ROGET Jarmasz (2003) Hassan and Mihalcea (2011) Knowledge-based 0.415 0.536
C&W Collobert and Weston (2008) Collobert and Weston (2008) Corpus-based 0.5 N/A
WikiRelate Strube and Ponzetto (2006) Strube and Ponzetto (2006) Corpus-based N/A 0.48
LSA Landauer et al. (1997) Hassan and Mihalcea (2011) Corpus-based 0.581 0.492
LSA Landauer et al. (1997) Hassan and Mihalcea (2011) Corpus-based 0.581 0.563
simVB+simWN Finkelstein et al. (2002) Finkelstein et al. (2002) Hybrid N/A 0.55
SSA Hassan and Mihalcea (2011) Hassan and Mihalcea (2011) Knowledge-based 0.622 0.629
HSMN+csmRNN Luong et al. (2013) Luong et al. (2013) Corpus-based 0.65 N/A
Multi-prototype Huang et al. (2012) Huang et al. (2012) Corpus-based 0.71 N/A
Multi-lingual SSA Hassan et al. (2011) Hassan et al. (2011) Corpus-based 0.713 0.674
ESA Gabrilovich and Markovitch (2007) Gabrilovich and Markovitch (2007) Corpus-based 0.748 0.503
TSA Radinsky et al. (2011) Radinsky et al. (2011) Hybrid 0.80 N/A
CLEAR Halawi et al. (2012) Halawi et al. (2012) Corpus-based 0.81 N/A