evaluation

Dear Colleagues,

the 4th Workshop on Evaluation and Comparison for NLP systems (Eval4NLP), co-located at the 2023 Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics (AACL 2023), invites the submission of long and short papers, with a theoretical or experimental nature, describing recent advances in system evaluation and comparison in NLP.

** Important Dates **

All deadlines are 11.59 pm UTC -12h (“Anywhere on Earth”).

ReproNLP 2023: First Call for Participation

Background

Dear all,

the “Test suites” sub-task will be included for the sixth time in the General MT Shared Task of the Conference on Machine Translation (WMT23).

*OVERVIEW*

Test suites are custom extensions to the test sets of the General MT Shared Task, constructed so that they can focus on particular aspects of the MT output. They cοnsist of a source-side test-set and a customized evaluation service. As opposed to the standard evaluation process which produces generic quality scores, test suites often produce separate fine-grained results for each phenomenon.

2nd Workshop on Novel Evaluation Approaches for Text Classification Systems

Co-located with ICWSM 2023, 5 June 2023, Limassol, Cyprus

https://neatclass-workshop.github.io/

The 3rd Workshop on Evaluation and Comparison for NLP systems (Eval4NLP, https://eval4nlp.github.io/2022), co-located at the 2022 Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics (AACL 2022, https://www.aacl2022.org/), invites the submission of long and short papers, with a theoretical or experimental nature, describing recent advances in system evaluation and comparison in NLP.

Fair evaluations and comparisons are of fundamental importance to the NLP community to properly track progress, especially within the current deep learning revolution, with new state-of-the-art results reported in ever shorter intervals. This concerns the creation of benchmark datasets that cover typical use cases and blind spots of existing systems, the designing of metrics for evaluating the performance of NLP systems on different dimensions, and the reporting of evaluation results in an unbiased manner.

The automatic or semiautomatic analysis of textual data is a key approach to analyse the massive amounts of user-generated content online, from the identification of sentiment in text and topic classification to the detection of abusive language, misinformation or propaganda. However, the development of such systems faces a crucial challenge.

--------------------------------------------
Second Call for Papers
--------------------------------------------
The 2nd Workshop on Evaluation and Comparison for NLP systems (Eval4NLP), co-located at the 2021 Conference on Empirical Methods in Natural Language Processing (EMNLP2021), invites the submission of long and short papers, with a theoretical or experimental nature, describing recent advances in system evaluation and comparison in NLP.

The GEM (Generation, Evaluation, Metrics) workshop at ACL, 2021 is inviting transformation submissions to NL-Augmenter.

--------------------------------------------
First Call for Papers
--------------------------------------------
The 2nd Workshop on Evaluation and Comparison for NLP systems (Eval4NLP), co-located at the 2021 Conference on Empirical Methods in Natural Language Processing (EMNLP2021), invites the submission of long and short papers, with a theoretical or experimental nature, describing recent advances in system evaluation and comparison in NLP.