Appraise Evaluation Framework for Machine Translation

Christian Federmann


Abstract
We present Appraise, an open-source framework for crowd-based annotation tasks, notably for evaluation of machine translation output. This is the software used to run the yearly evaluation campaigns for shared tasks at the WMT Conference on Machine Translation. It has also been used at IWSLT 2017 and, recently, to measure human parity for machine translation for Chinese to English news text. The demo will present the full end-to-end lifecycle of an Appraise evaluation campaign, from task creation to annotation and interpretation of results.
Anthology ID:
C18-2019
Volume:
Proceedings of the 27th International Conference on Computational Linguistics: System Demonstrations
Month:
August
Year:
2018
Address:
Santa Fe, New Mexico
Editor:
Dongyan Zhao
Venue:
COLING
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
86–88
Language:
URL:
https://aclanthology.org/C18-2019
DOI:
Bibkey:
Cite (ACL):
Christian Federmann. 2018. Appraise Evaluation Framework for Machine Translation. In Proceedings of the 27th International Conference on Computational Linguistics: System Demonstrations, pages 86–88, Santa Fe, New Mexico. Association for Computational Linguistics.
Cite (Informal):
Appraise Evaluation Framework for Machine Translation (Federmann, COLING 2018)
Copy Citation:
PDF:
https://aclanthology.org/C18-2019.pdf