Difference between revisions of "SemEval-2012 Task 2 (State of the art)"
(5 intermediate revisions by the same user not shown) | |||
Line 16: | Line 16: | ||
! [http://en.wikipedia.org/wiki/MaxDiff MaxDiff] | ! [http://en.wikipedia.org/wiki/MaxDiff MaxDiff] | ||
! [http://en.wikipedia.org/wiki/Spearman%27s_rank_correlation_coefficient Spearman] | ! [http://en.wikipedia.org/wiki/Spearman%27s_rank_correlation_coefficient Spearman] | ||
+ | |- | ||
+ | | BUAP | ||
+ | | Tovar et al. (2012) | ||
+ | | 31.7 | ||
+ | | 0.014 | ||
|- | |- | ||
| Random baseline | | Random baseline | ||
Line 21: | Line 26: | ||
| 31.2 | | 31.2 | ||
| 0.018 | | 0.018 | ||
− | |||
− | |||
− | |||
− | |||
− | |||
|- | |- | ||
| Duluth-V2 | | Duluth-V2 | ||
Line 66: | Line 66: | ||
| ---- | | ---- | ||
| 0.334 | | 0.334 | ||
+ | |- | ||
+ | | PairDirection | ||
+ | | Levy & Goldberg (2014) | ||
+ | | 45.2 | ||
+ | | ---- | ||
|- | |- | ||
| Com | | Com | ||
Line 80: | Line 85: | ||
== References == | == References == | ||
+ | |||
+ | * '''Listed alphabetically.''' | ||
+ | |||
Isaac I. Bejar, Roger Chaffin, and Susan E. Embretson. (1991). ''[http://www.amazon.com/Cognitive-Psychometric-Analysis-Analogical-Psychology/dp/0387973214/ Cognitive and Psychometric Analysis of Analogical Problem Solving]''. Springer-Verlag. | Isaac I. Bejar, Roger Chaffin, and Susan E. Embretson. (1991). ''[http://www.amazon.com/Cognitive-Psychometric-Analysis-Analogical-Psychology/dp/0387973214/ Cognitive and Psychometric Analysis of Analogical Problem Solving]''. Springer-Verlag. | ||
David A. Jurgens, Saif M. Mohammad, Peter D. Turney, and Keith J. Holyoak. (2012). [http://ixa2.si.ehu.es/starsem/proc/pdf/STARSEM-SEMEVAL047.pdf SemEval-2012 Task 2: Measuring degrees of relational similarity]. In ''Proceedings of the First Joint Conference on Lexical and Computational Semantics (*SEM)'', pages 356-364, Montreal, Canada. | David A. Jurgens, Saif M. Mohammad, Peter D. Turney, and Keith J. Holyoak. (2012). [http://ixa2.si.ehu.es/starsem/proc/pdf/STARSEM-SEMEVAL047.pdf SemEval-2012 Task 2: Measuring degrees of relational similarity]. In ''Proceedings of the First Joint Conference on Lexical and Computational Semantics (*SEM)'', pages 356-364, Montreal, Canada. | ||
+ | |||
+ | Omer Levy and Yoav Goldberg (2014). [http://www.cs.bgu.ac.il/~yoavg/publications/conll2014analogies.pdf Linguistic Regularities in Sparse and Explicit Word Representations], In ''Proceedings of the Eighteenth Conference on Computational Natural Language Learning (CONLL 2014)'', Baltimore, Maryland. | ||
Tomas Mikolov, Wen-tau Yih, and Geoffrey Zweig. (2013). [http://research.microsoft.com/pubs/189726/rvecs.pdf Linguistic regularities in continuous space word representations]. In ''Proceedings of the 2013 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL 2013)'', Atlanta, Georgia. | Tomas Mikolov, Wen-tau Yih, and Geoffrey Zweig. (2013). [http://research.microsoft.com/pubs/189726/rvecs.pdf Linguistic regularities in continuous space word representations]. In ''Proceedings of the 2013 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL 2013)'', Atlanta, Georgia. | ||
Line 97: | Line 107: | ||
Mireya Tovar, David Pinto, Azucena Montes, and Darnes Vilariño (2013). [http://link.springer.com/chapter/10.1007/978-3-642-38989-4_37 Determining the degree of semantic similarity using prototype vectors]. Pattern Recognition, Volume 7914, 364-373. | Mireya Tovar, David Pinto, Azucena Montes, and Darnes Vilariño (2013). [http://link.springer.com/chapter/10.1007/978-3-642-38989-4_37 Determining the degree of semantic similarity using prototype vectors]. Pattern Recognition, Volume 7914, 364-373. | ||
− | Peter D. Turney. (2013), [http://aclweb.org/anthology | + | Peter D. Turney. (2013), [http://aclweb.org/anthology/Q/Q13/Q13-1029.pdf Distributional semantics beyond words: Supervised learning of analogy and paraphrase], In ''Transactions of the Association for Computational Linguistics (TACL)'', 1, 353-366. |
Alisa Zhila, Wen-tau Yih, Christopher Meek, Geoffrey Zweig, and Tomas Mikolov. (2013). [http://research.microsoft.com/pubs/189727/RelationalSimilarity.pdf Combining heterogeneous models for measuring relational similarity]. In ''Proceedings of the 2013 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL 2013)'', Atlanta, Georgia. | Alisa Zhila, Wen-tau Yih, Christopher Meek, Geoffrey Zweig, and Tomas Mikolov. (2013). [http://research.microsoft.com/pubs/189727/RelationalSimilarity.pdf Combining heterogeneous models for measuring relational similarity]. In ''Proceedings of the 2013 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL 2013)'', Atlanta, Georgia. | ||
Line 103: | Line 113: | ||
[[Category:State of the art]] | [[Category:State of the art]] | ||
+ | [[Category:Similarity]] |
Revision as of 11:25, 28 June 2015
- SemEval-2012 Task 2: Measuring Degrees of Relational Similarity
- graded relational similarity ratings across 79 relation categories
- relation taxonomy of Bejar et al. (1991)
- see also: Similarity (State of the art)
Table of results
- Listed in order of increasing Spearman's rho.
Algorithm | Reference | MaxDiff | Spearman |
---|---|---|---|
BUAP | Tovar et al. (2012) | 31.7 | 0.014 |
Random baseline | Jurgens et al. (2012) | 31.2 | 0.018 |
Duluth-V2 | Pedersen (2012) | 31.1 | 0.038 |
Duluth-V1 | Pedersen (2012) | 31.5 | 0.039 |
Duluth-V0 | Pedersen (2012) | 32.4 | 0.050 |
PMI baseline | Jurgens et al. (2012) | 33.9 | 0.112 |
UTD-SVM | Rink & Harabagiu (2012) | 34.7 | 0.116 |
UTD-NB | Rink & Harabagiu (2012) | 39.4 | 0.229 |
RNN-1600 | Mikolov et al. (2013) | 41.8 | 0.275 |
UTD-LDA | Rink & Harabagiu (2013) | ---- | 0.334 |
PairDirection | Levy & Goldberg (2014) | 45.2 | ---- |
Com | Zhila et al. (2013) | 45.2 | 0.353 |
SuperSim | Turney (2013) | 47.2 | 0.408 |
References
- Listed alphabetically.
Isaac I. Bejar, Roger Chaffin, and Susan E. Embretson. (1991). Cognitive and Psychometric Analysis of Analogical Problem Solving. Springer-Verlag.
David A. Jurgens, Saif M. Mohammad, Peter D. Turney, and Keith J. Holyoak. (2012). SemEval-2012 Task 2: Measuring degrees of relational similarity. In Proceedings of the First Joint Conference on Lexical and Computational Semantics (*SEM), pages 356-364, Montreal, Canada.
Omer Levy and Yoav Goldberg (2014). Linguistic Regularities in Sparse and Explicit Word Representations, In Proceedings of the Eighteenth Conference on Computational Natural Language Learning (CONLL 2014), Baltimore, Maryland.
Tomas Mikolov, Wen-tau Yih, and Geoffrey Zweig. (2013). Linguistic regularities in continuous space word representations. In Proceedings of the 2013 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL 2013), Atlanta, Georgia.
Ted Pedersen (2012), Duluth: Measuring Degrees of Relational Similarity with the Gloss Vector Measure of Semantic Relatedness, In Proceedings of the First Joint Conference on Lexical and Computational Semantics (*SEM), Montreal, Canada, pp. 497-501.
Bryan Rink and Sanda Harabagiu (2012), UTD: Determining Relational Similarity Using Lexical Patterns, In Proceedings of the First Joint Conference on Lexical and Computational Semantics (*SEM), Montreal, Canada, pp. 413-418.
Bryan Rink and Sanda Harabagiu. (2013). The impact of selectional preference agreement on semantic relational similarity. In Proceedings of the 10th International Conference on Computational Semantics (IWCS 2013), Potsdam, Germany.
Mireya Tovar, J. Alejandro Reyes, Azucena Montes, Darnes Vilariño, David Pinto, and Saul León (2012), BUAP: A First Approximation to Relational Similarity Measuring, In Proceedings of the First Joint Conference on Lexical and Computational Semantics (*SEM), Montreal, Canada, pp. 502-505.
Mireya Tovar, David Pinto, Azucena Montes, and Darnes Vilariño (2013). Determining the degree of semantic similarity using prototype vectors. Pattern Recognition, Volume 7914, 364-373.
Peter D. Turney. (2013), Distributional semantics beyond words: Supervised learning of analogy and paraphrase, In Transactions of the Association for Computational Linguistics (TACL), 1, 353-366.
Alisa Zhila, Wen-tau Yih, Christopher Meek, Geoffrey Zweig, and Tomas Mikolov. (2013). Combining heterogeneous models for measuring relational similarity. In Proceedings of the 2013 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL 2013), Atlanta, Georgia.