Difference between revisions of "Noun-Modifier Questions (State of the art)"

From ACL Wiki
Jump to: navigation, search
Line 4: Line 4:
 
* split into 680 questions for training (e.g., development, parameter tuning) and 1,500 questions for testing
 
* split into 680 questions for training (e.g., development, parameter tuning) and 1,500 questions for testing
 
* introduced in [http://jair.org/papers/paper3640.html Turney (2012)]
 
* introduced in [http://jair.org/papers/paper3640.html Turney (2012)]
 +
* see also: [[Similarity (State of the art)]]
  
 
== Sample question ==
 
== Sample question ==
Line 63: Line 64:
 
|-  
 
|-  
 
| Element-wise multiplication
 
| Element-wise multiplication
|  
+
| Mitchell and Lapata (2008)
|  
+
| Turney (2012)
 
| 57.5
 
| 57.5
 
| 27.4
 
| 27.4
 
|-
 
|-
 
| Dual-Space model
 
| Dual-Space model
|
+
| Turney (2012)
|
+
| Turney (2012)
 
| 58.3  
 
| 58.3  
 
| 41.5
 
| 41.5
 
|-
 
|-
 
| SuperSim
 
| SuperSim
|  
+
| Turney (2013)
|  
+
| Turney (2013)
 
| 75.9
 
| 75.9
 
| 68.0
 
| 68.0
 
|-
 
|-
 
| Holistic model
 
| Holistic model
|  
+
| Turney (2012)
|  
+
| Turney (2012)
 
| 81.6
 
| 81.6
 
| NA
 
| NA

Revision as of 05:22, 28 April 2014

Sample question

Stem: coin collector
Choices: (a) numismatist
(b) coin
(c) collector
(d) coinage
(e) aggregator
(f) reflectiveness
(g) plasma
Solution: (a) numismatist

Table of results

Algorithm Reference for algorithm Reference for experiment Correct - 7-choices Correct - 14-choices
Vector addition Landauer and Dumais (1997) Turney (2012) 50.1 22.5
Element-wise multiplication Mitchell and Lapata (2008) Turney (2012) 57.5 27.4
Dual-Space model Turney (2012) Turney (2012) 58.3 41.5
SuperSim Turney (2013) Turney (2013) 75.9 68.0
Holistic model Turney (2012) Turney (2012) 81.6 NA

References

Turney, P.D. (2012). Domain and function: A dual-space model of semantic relations and compositions, Journal of Artificial Intelligence Research (JAIR), 44, 533-585.

Turney, P.D. (2013), Distributional semantics beyond words: Supervised learning of analogy and paraphrase, Transactions of the Association for Computational Linguistics (TACL), 1, 353-366.