Team Papelo: Transformer Networks at FEVER

Christopher Malon


Abstract
We develop a system for the FEVER fact extraction and verification challenge that uses a high precision entailment classifier based on transformer networks pretrained with language modeling, to classify a broad set of potential evidence. The precision of the entailment classifier allows us to enhance recall by considering every statement from several articles to decide upon each claim. We include not only the articles best matching the claim text by TFIDF score, but read additional articles whose titles match named entities and capitalized expressions occurring in the claim text. The entailment module evaluates potential evidence one statement at a time, together with the title of the page the evidence came from (providing a hint about possible pronoun antecedents). In preliminary evaluation, the system achieves .5736 FEVER score, .6108 label accuracy, and .6485 evidence F1 on the FEVER shared task test set.
Anthology ID:
W18-5517
Volume:
Proceedings of the First Workshop on Fact Extraction and VERification (FEVER)
Month:
November
Year:
2018
Address:
Brussels, Belgium
Editors:
James Thorne, Andreas Vlachos, Oana Cocarascu, Christos Christodoulopoulos, Arpit Mittal
Venue:
EMNLP
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
109–113
Language:
URL:
https://aclanthology.org/W18-5517
DOI:
10.18653/v1/W18-5517
Bibkey:
Cite (ACL):
Christopher Malon. 2018. Team Papelo: Transformer Networks at FEVER. In Proceedings of the First Workshop on Fact Extraction and VERification (FEVER), pages 109–113, Brussels, Belgium. Association for Computational Linguistics.
Cite (Informal):
Team Papelo: Transformer Networks at FEVER (Malon, EMNLP 2018)
Copy Citation:
PDF:
https://aclanthology.org/W18-5517.pdf
Code
 cdmalon/finetune-transformer-lm
Data
FEVERSNLI