CoMeDi Shared Task: Median Judgment Classification & Mean Disagreement Ranking with Ordinal Word-in-Context Judgments

The CoMeDi (Context and Meaning—Navigating Disagreements in NLP Annotations) workshop will host a shared task. We invite participants to solve two subtasks given a pair of words uses:

Subtask 1: Ordinal Word-in-Context Classification (OGWiC)
Subtask 2: Disagreement in Word-in-Context Ranking (DisWiC)

The task takes a different view on modeling of word meaning by (i) treating WiC as an ordinal classification task, and (ii) making disagreement the explicit detection aim (instead of removing it).

The workshop will colocated with COLING 2025, where the results of the shared task will be presented. All information can be found on our website.