Difference between revisions of "DIRT Paraphrase Collection - RTE Users"
Jump to navigation
Jump to search
| (6 intermediate revisions by the same user not shown) | |||
| Line 1: | Line 1: | ||
| − | |||
<br /> | <br /> | ||
| − | {|class="wikitable sortable" cellpadding="3" cellspacing="0" border="1 | + | {|class="wikitable sortable" cellpadding="3" cellspacing="0" border="1" |
|- bgcolor="#CDCDCD" | |- bgcolor="#CDCDCD" | ||
! nowrap="nowrap"|Participants* | ! nowrap="nowrap"|Participants* | ||
| Line 15: | Line 14: | ||
| We used the canonical DIRT rulebase version of Szpektor and Dagan (RANLP 2007), and considered top 25 rules. | | We used the canonical DIRT rulebase version of Szpektor and Dagan (RANLP 2007), and considered top 25 rules. | ||
| Ablation test performed. Positive impact of the resource: +1.33% accuracy on two-way task. | | Ablation test performed. Positive impact of the resource: +1.33% accuracy on two-way task. | ||
| + | |||
| + | |- bgcolor="#ECECEC" align="left" | ||
| + | | Boeing | ||
| + | | RTE5 | ||
| + | | | ||
| + | | Verb paraphrases | ||
| + | | Ablation test performed. Negative impact of the resource on two-way task: -1.17% accuracy. Null impact of the resource on three-way task. | ||
| + | |||
| + | |- bgcolor="#ECECEC" align="left" | ||
| + | | UAIC | ||
| + | | RTE5 | ||
| + | | | ||
| + | | Use of DIRT relations to map verbs in T with verbs in H | ||
| + | | Ablation test performed. Positive impact of the resource: +0.17% accuracy on two-way, +0.33% on three-way task. | ||
|- bgcolor="#ECECEC" align="left" | |- bgcolor="#ECECEC" align="left" | ||
| Line 36: | Line 49: | ||
| RTE4 | | RTE4 | ||
| | | | ||
| − | | | + | | Use of DIRT relations to map verbs in T with verbs in H |
| − | | | + | | Ablation test performed: +0.7% precision on two-way task. |
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
|- bgcolor="#ECECEC" align="left" | |- bgcolor="#ECECEC" align="left" | ||
| Line 54: | Line 60: | ||
|- bgcolor="#ECECEC" align="left" | |- bgcolor="#ECECEC" align="left" | ||
| − | | | + | | UAIC |
| + | | RTE3 | ||
| | | | ||
| + | | Use of DIRT relations to map verbs in T with verbs in H | ||
| + | | Ablation test performed: +0.37% precision on two-way task. | ||
| + | |||
| + | |- bgcolor="#ECECEC" align="left" | ||
| + | | UIUC | ||
| + | | RTE3 | ||
| | | | ||
| + | | Paired verb/argument patterns | ||
| | | | ||
| − | |||
|} | |} | ||
{|class="wikitable" cellpadding="3" cellspacing="0" border="0" style="margin-left: 20px;" | {|class="wikitable" cellpadding="3" cellspacing="0" border="0" style="margin-left: 20px;" | ||
|- | |- | ||
| − | ! align="left"|Total: | + | ! align="left"|Total: 9 |
|} | |} | ||
<br> | <br> | ||
Latest revision as of 10:32, 22 December 2009
| Participants* | Campaign | Version | Specific usage description | Evaluations / Comments |
|---|---|---|---|---|
| BIU | RTE5 | We used the canonical DIRT rulebase version of Szpektor and Dagan (RANLP 2007), and considered top 25 rules. | Ablation test performed. Positive impact of the resource: +1.33% accuracy on two-way task. | |
| Boeing | RTE5 | Verb paraphrases | Ablation test performed. Negative impact of the resource on two-way task: -1.17% accuracy. Null impact of the resource on three-way task. | |
| UAIC | RTE5 | Use of DIRT relations to map verbs in T with verbs in H | Ablation test performed. Positive impact of the resource: +0.17% accuracy on two-way, +0.33% on three-way task. | |
| BIU | RTE4 | We used the canonical DIRT rulebase version of Szpektor and Dagan (RANLP 2007), and considered top 25 rules. | +0.9% on RTE-4 ablation tests | |
| Boeing | RTE4 | Original DIRT db | Elaborate T sentence with DIRT-implied entailments | precision/recall in RTE4: boeing run1: 67%/6%; |
| UAIC | RTE4 | Use of DIRT relations to map verbs in T with verbs in H | Ablation test performed: +0.7% precision on two-way task. | |
| Uoeltg | RTE4 | Data taken from the RTE4 proceedings. Participants are recommended to add further information. | ||
| UAIC | RTE3 | Use of DIRT relations to map verbs in T with verbs in H | Ablation test performed: +0.37% precision on two-way task. | |
| UIUC | RTE3 | Paired verb/argument patterns |
| Total: 9 |
|---|
[*] For further information about participants, click here: RTE Challenges - Data about participants
Return to RTE Knowledge Resources