The 1st Evaluation of Multi-Modal Generation Workshop @ COLING 2025
----------------------------------------------------------------------------------------------------------
Multimodal generation techniques have opened new avenues for creative content generation. However, evaluating the quality of multimodal generation remains underexplored and some key questions are unanswered, such as the contributions of each modal, the utility of pre-trained large language models for multimodal generation, and measuring faithfulness and fairness in multimodal outputs. This workshop aims to foster discussions and research efforts by bringing together researchers and practitioners in natural language processing, computer vision, and multimodal AI. Our goal is to establish evaluation methods for multimodal research and advance research efforts in this direction.
Call for Papers
----------------------
Both long paper and short papers (up to 8 pages and 4 pages respectively with unlimited references and appendices) are welcomed for submission.
A list of topics relevant to this workshop (but not limited to):
- Evaluation metrics for multimodal text generation for assessing informativeness, factuality and faithfulness
- New benchmark datasets, evaluation protocols and annotations
- Challenges in evaluating multimodal coherence, relevance and contribution of modalities and inter- and intra-interactions
- Assessing information integration and aggregation across multiple modalities
- Adversarial evaluation approaches for testing the robustness and reliability of multimodal generation systems
- Ethical considerations in the evaluation of multimodal text generation, including bias detection and mitigation strategies
- Multilingual multimodal text generation systems for low-resource languages
- Evaluating fairness and privacy in multimodal learning and applications
Important Dates
---------------------------
- Nov 20, 2024: Paper submission due date
- Dec 05, 2024: Notification of acceptance
- Dec 11, 2024: Camera-ready version due
- Jan 19, 2025: Workshop Date
Note: All deadlines are 11:59PM UTC-12:00 (“Anywhere on Earth”)
Submission Instructions
-------------------------------------
You are invited to submit your papers in our START/SoftConf submission portal. All the submitted papers have to be anonymous for double-blind review. The content of the paper should not be longer than 8 pages for long papers and 4 pages for short papers, strictly following the ACL style templates, with the mandatory limitation section not counting towards the page limit. Supplementary and appendices (either as separate files or appended after the main submission) are allowed. We encourage code link submissions for reproducibility.
Submission Link: https://softconf.com/coling2025/EvalMG25
ACL style template: https://aclrollingreview.org/cfp#paper-submission-and-templates
Non-archival Option
-------------------------------
To promote discussions within the community, our workshop includes non-archival track. Authors have the flexbility to submit their unpublished work or papers accepted to COLING main conference to our workshop. The organisers may offer the opportunity to give oral or poster presentation.