WordNet - RTE Users

When not otherwise specified, the data about version, usage and evaluation of the resource have been provided by participants themselves.

Participants*	Campaign	Version	Specific usage description	Evaluations / Comments
AUEB	RTE5		During the calculation of the similarity measures we treat words from T and H that are synonyms according to WordNet as identical.	Ablation test performed. Negative impact of the resource: -2% accuracy on two-way, -2.67% on three-way task.
BIU	RTE5	3.0	Synonyms, hyponyms (2 levels away from the original term), the hyponym_instance relation and derivations.	Ablation test performed. Positive impact of the resource: +2.5% accuracy on two-way task.
Boeing	RTE5		The system makes uses Wordnet synonyms, hypernyms relationships between (senses of) words, "similar" (SIM), "pertains" (PER), and "derivational" (DER) links to recognize equivalence between T and H.	Ablation test performed. Positive impact of the resource: +4% accuracy on two-way, +5.67% on three-way task.
DFKI	RTE5		FIRST USE: Argument alignment between T and H. SECOND USE: used to change all the nominal predicates into verbs, to calculate relatedness between T and H (using VerbOcean).	FIRST USE: Ablation test performed. Impact of the resource: -0.17% accuracy/null respectively on two-way and three-way task for run1; +0.16%/+0.34% for run2; +0.17%/+0.17% for run3. SECOND USE (WordNet+VerbOcean): null/+0.17% accuracy respectively on two-way and three-way task for run1; +0.5%/+0.67% for run2; +0.17%/+0.17% for run3.
DirRelCond	RTE5		Use of many WordNet relations (such as synonymy, hypernymy, hyponymy, meronymy, holonymy etc.) to compute the relatedness between words with the same part of speech in T and H.	No ablation test performed. The resource cannot be removed without breaking the functionality of the system.
DLSIUAES	RTE5		FIRST USE: Similarity between lemmata, computed by WordNet-based metrics. SECOND USE: antonymy relations between verbs.	FIRST USE: Ablation test performed. Positive impact of the resource on two-way run: +0.83% accuracy. Negative impact on three-way run: -0.33% accuracy (-0.5% for two-way derived). SECOND USE (WordNet+VerbOcean+DLSIUAES_negation_list): positive impact on two-way run: +0.66% accuracy. Negative impact on three-way run: -1% (-0.5% for two-way derived).
AUEB	RTE4			Data taken from the RTE4 proceedings. Participants are recommended to add further information.
BIU	RTE4	3.0	Synonyms, hyponyms (2 levels away from the original term), the hyponym_instance relation and derivations. Also used as part of our novel lexical-syntactic resource	0.8% improvement in ablation test on RTE-4. Potential contribution is higher since this resource partially overlaps with the novel lexical-syntactic rule base
Boeing	RTE4	2.0	Semantic relation between words	No formal evaluation. Plays a role in most entailments found
Cambridge	RTE4	3.0	Meaning postulates from WordNet noun hyponymy, e.g. forall x: cat(x) -> animal(x)	No systematic evaluation
CERES	RTE4	3.0	Hypernyms, antonyms, indexWords (N,V,Adj,Adv)	Used, but no evaluation performed
DFKI	RTE4	3.0	Semantic relation between words	No separate evaluation
DLSIUAES	RTE4			Data taken from the RTE4 proceedings. Participants are recommended to add further information.
EMORY	RTE4			Data taken from the RTE4 proceedings. Participants are recommended to add further information.
FbkIrst	RTE4	3.0	Lexical similarity	No precise evaluation of the resource has been carried out. In our second run we used a combined system (EDITSneg + EDITSallbutneg), and we had an improvement of 0.6% in accuracy with respect to the first run in which only EDITSneg was used. EDITSallbutneg exploits lexical similarity (WordNet similarity), but we can’t affirm with precision that the improvement is due only to the use of WordNet
FSC	RTE4			Data taken from the RTE4 proceedings. Participants are recommended to add further information.
IIT	RTE4			Data taken from the RTE4 proceedings. Participants are recommended to add further information.
IPD	RTE4			Data taken from the RTE4 proceedings. Participants are recommended to add further information.
OAQA	RTE4			Data taken from the RTE4 proceedings. Participants are recommended to add further information.
QUANTA	RTE4			Data taken from the RTE4 proceedings. Participants are recommended to add further information.
SAGAN	RTE4			Data taken from the RTE4 proceedings. Participants are recommended to add further information.
Stanford	RTE4			Data taken from the RTE4 proceedings. Participants are recommended to add further information.
UAIC	RTE4			Data taken from the RTE4 proceedings. Participants are recommended to add further information.
UMD	RTE4			Data taken from the RTE4 proceedings. Participants are recommended to add further information.
UNED	RTE4			Data taken from the RTE4 proceedings. Participants are recommended to add further information.
Uoeltg	RTE4			Data taken from the RTE4 proceedings. Participants are recommended to add further information.
UPC	RTE4			Data taken from the RTE4 proceedings. Participants are recommended to add further information.
AUEB	RTE3	2.1	Synonymy resolution	Replacing the words of H with their synonyms in T: on RTE3 data sets 2% improvement
UIUC	RTE3		Semantic distance between words
VENSES	RTE3	3.0	Semantic relation between words	No evaluation of the resource
New user				Participants are encouraged to contribute.

Total: 24

[*] For further information about participants, click here: RTE Challenges - Data about participants

   Return to RTE Knowledge Resources