Difference between revisions of "SemEval-2012 Task 2 (State of the art)"

From ACL Wiki
Jump to: navigation, search
(Created page with "* [https://sites.google.com/site/semeval2012task2/home SemEval-2012 Task 2]: Measuring Degrees of Relational Similarity * graded relational similarity ratings across 79 relati...")
 
Line 5: Line 5:
  
 
== Table of results ==  
 
== Table of results ==  
 +
 +
* '''Listed in order of increasing [http://en.wikipedia.org/wiki/Spearman_rank_correlation Spearman's rho].'''
 +
  
 
{| border="1" cellpadding="5" cellspacing="1"
 
{| border="1" cellpadding="5" cellspacing="1"
Line 10: Line 13:
 
! Algorithm
 
! Algorithm
 
! Reference
 
! Reference
! MaxDiff
+
! [http://en.wikipedia.org/wiki/MaxDiff MaxDiff]
! Spearman
+
! [http://en.wikipedia.org/wiki/Spearman%27s_rank_correlation_coefficient Spearman]
 
|-
 
|-
 
| Random
 
| Random
Line 76: Line 79:
  
 
== References ==
 
== References ==
 +
 +
Isaac I. Bejar, Roger Chaffin, and Susan E. Embretson. (1991). ''[http://www.amazon.com/Cognitive-Psychometric-Analysis-Analogical-Psychology/dp/0387973214/ Cognitive and Psychometric Analysis of Analogical Problem Solving]''. Springer-Verlag.
  
 
David A. Jurgens, Saif M. Mohammad, Peter D. Turney, and Keith J. Holyoak. (2012). [http://ixa2.si.ehu.es/starsem/proc/pdf/STARSEM-SEMEVAL047.pdf SemEval-2012 Task 2: Measuring degrees of relational similarity]. In ''Proceedings of the First Joint Conference on Lexical and Computational Semantics (*SEM)'', pages 356-364, Montreal, Canada.
 
David A. Jurgens, Saif M. Mohammad, Peter D. Turney, and Keith J. Holyoak. (2012). [http://ixa2.si.ehu.es/starsem/proc/pdf/STARSEM-SEMEVAL047.pdf SemEval-2012 Task 2: Measuring degrees of relational similarity]. In ''Proceedings of the First Joint Conference on Lexical and Computational Semantics (*SEM)'', pages 356-364, Montreal, Canada.

Revision as of 09:17, 15 October 2013

  • SemEval-2012 Task 2: Measuring Degrees of Relational Similarity
  • graded relational similarity ratings across 79 relation categories
  • relation taxonomy of Bejar et al. (1991)


Table of results


Algorithm Reference MaxDiff Spearman
Random Jurgens et al. (2012) 31.2 0.018
BUAP Tovar et al. (2012) 31.7 0.014
Duluth-V2 Pedersen (2012) 31.1 0.038
Duluth-V1 Pedersen (2012) 31.5 0.039
Duluth-V0 Pedersen (2012) 32.4 0.050
PMI Jurgens et al. (2012) 33.9 0.112
UTD-SVM Rink & Harabagiu (2012) 34.7 0.116
UTD-NB Rink & Harabagiu (2012) 39.4 0.229
RNN-1600 Mikolov et al. (2013) 0.275
UTD-LDA Rink & Harabagiu (2013) 0.334
Com Zhila et al. (2013) 0.353
SuperSim Turney (2013) 0.408


References

Isaac I. Bejar, Roger Chaffin, and Susan E. Embretson. (1991). Cognitive and Psychometric Analysis of Analogical Problem Solving. Springer-Verlag.

David A. Jurgens, Saif M. Mohammad, Peter D. Turney, and Keith J. Holyoak. (2012). SemEval-2012 Task 2: Measuring degrees of relational similarity. In Proceedings of the First Joint Conference on Lexical and Computational Semantics (*SEM), pages 356-364, Montreal, Canada.

Ted Pedersen (2012), Duluth: Measuring Degrees of Relational Similarity with the Gloss Vector Measure of Semantic Relatedness, In Proceedings of the First Joint Conference on Lexical and Computational Semantics (*SEM), Montreal, Canada, pp. 497-501.

Bryan Rink and Sanda Harabagiu (2012), UTD: Determining Relational Similarity Using Lexical Patterns, In Proceedings of the First Joint Conference on Lexical and Computational Semantics (*SEM), Montreal, Canada, pp. 413-418.

Mireya Tovar, J. Alejandro Reyes, Azucena Montes, Darnes Vilariño, David Pinto, and Saul León (2012), BUAP: A First Approximation to Relational Similarity Measuring, In Proceedings of the First Joint Conference on Lexical and Computational Semantics (*SEM), Montreal, Canada, pp. 502-505.


See also