Difference between revisions of "DIRT Paraphrase Collection - RTE Users"
Jump to navigation
Jump to search
| Line 8: | Line 8: | ||
! nowrap="nowrap"|Specific usage description | ! nowrap="nowrap"|Specific usage description | ||
! nowrap="nowrap"|Evaluations / Comments | ! nowrap="nowrap"|Evaluations / Comments | ||
| + | |||
|- bgcolor="#ECECEC" align="left" | |- bgcolor="#ECECEC" align="left" | ||
| − | | | + | | BIU |
| − | | | + | | RTE5 |
| | | | ||
| − | | | + | | We used the canonical DIRT rulebase version of Szpektor and Dagan (RANLP 2007), and considered top 25 rules. |
| + | | Ablation test performed. Positive impact of the resource: +1.33% accuracy on two-way task. | ||
| + | |||
| + | |- bgcolor="#ECECEC" align="left" | ||
| + | | BIU | ||
| + | | RTE4 | ||
| | | | ||
| + | | We used the canonical DIRT rulebase version of Szpektor and Dagan (RANLP 2007), and considered top 25 rules. | ||
| + | | +0.9% on RTE-4 ablation tests | ||
| + | |||
|- bgcolor="#ECECEC" align="left" | |- bgcolor="#ECECEC" align="left" | ||
| Boeing | | Boeing | ||
| Line 22: | Line 31: | ||
boeing run1: 67%/6%;<br> | boeing run1: 67%/6%;<br> | ||
boeing run2: 54%/30% | boeing run2: 54%/30% | ||
| + | |||
|- bgcolor="#ECECEC" align="left" | |- bgcolor="#ECECEC" align="left" | ||
| − | | | + | | UAIC |
| RTE4 | | RTE4 | ||
| | | | ||
| − | | | + | | |
| − | + | | ''Data taken from the RTE4 proceedings. Participants are recommended to add further information.'' | |
| + | |||
|- bgcolor="#ECECEC" align="left" | |- bgcolor="#ECECEC" align="left" | ||
| − | | | + | | UIUC |
| − | | | + | | RTE3 |
| | | | ||
| + | | Paired verb/argument patterns | ||
| | | | ||
| − | + | ||
|- bgcolor="#ECECEC" align="left" | |- bgcolor="#ECECEC" align="left" | ||
| Uoeltg | | Uoeltg | ||
| Line 40: | Line 52: | ||
| | | | ||
| ''Data taken from the RTE4 proceedings. Participants are recommended to add further information.'' | | ''Data taken from the RTE4 proceedings. Participants are recommended to add further information.'' | ||
| + | |||
|- bgcolor="#ECECEC" align="left" | |- bgcolor="#ECECEC" align="left" | ||
| ''New user'' | | ''New user'' | ||
| Line 46: | Line 59: | ||
| | | | ||
| ''Participants are encouraged to contribute.'' | | ''Participants are encouraged to contribute.'' | ||
| + | |||
|} | |} | ||
{|class="wikitable" cellpadding="3" cellspacing="0" border="0" style="margin-left: 20px;" | {|class="wikitable" cellpadding="3" cellspacing="0" border="0" style="margin-left: 20px;" | ||
Revision as of 08:00, 26 November 2009
When not otherwise specified, the data about version, usage and evaluation of the resource have been provided by participants themselves.
| Participants* | Campaign | Version | Specific usage description | Evaluations / Comments |
|---|---|---|---|---|
| BIU | RTE5 | We used the canonical DIRT rulebase version of Szpektor and Dagan (RANLP 2007), and considered top 25 rules. | Ablation test performed. Positive impact of the resource: +1.33% accuracy on two-way task. | |
| BIU | RTE4 | We used the canonical DIRT rulebase version of Szpektor and Dagan (RANLP 2007), and considered top 25 rules. | +0.9% on RTE-4 ablation tests | |
| Boeing | RTE4 | Original DIRT db | Elaborate T sentence with DIRT-implied entailments | precision/recall in RTE4: boeing run1: 67%/6%; |
| UAIC | RTE4 | Data taken from the RTE4 proceedings. Participants are recommended to add further information. | ||
| UIUC | RTE3 | Paired verb/argument patterns | ||
| Uoeltg | RTE4 | Data taken from the RTE4 proceedings. Participants are recommended to add further information. | ||
| New user | Participants are encouraged to contribute. |
| Total: 5 |
|---|
[*] For further information about participants, click here: RTE Challenges - Data about participants
Return to RTE Knowledge Resources