Please consider contributing and/or forwarding to appropriate colleagues and groups.
****We apologize for the multiple copies of this e-mail****
First Call for Participation - EXIST 2024 at CLEF 2024
Task: EXIST 2024: sEXism Identification in Social neTworks
Website: http://nlp.uned.es/exist2024/
EXIST is a series of scientific events and shared tasks on sEXism Identification in Social neTworks (EXIST 2021, EXIST 2022, EXIST 2023). EXIST aims to capture sexism in a broad sense, from explicit misogyny to other subtle expressions that involve implicit sexist behaviors. The fourth edition of the EXIST shared task will be held as a Lab in CLEF 2024, on September 9-12, 2024, at the University of Grenoble Alpes, France.
Since 2021, the primary objective of EXIST campaigns has been the identification of sexism in tweets. Three corpora of annotated tweets have been collected for different EXIST tasks. Likewise, the focus of EXIST 2024 is to detect sexism in text, using the EXIST 2023 dataset, but we also extend the focus to memes. Memes are images, usually with text captions, that typically carry humor and spread through social media, forums, or other digital platforms. They can be used to spread false information, perpetuate stereotypes or humiliate people.
As in the 2023 edition, this edition will also embrace the Learning With Disagreement (LeWiDi) paradigm for both the development of the dataset and the evaluation of the systems. The LeWiDi paradigm does not rely on a single “correct” label for each example. Instead, the model is trained to handle and learn from conflicting or diverse annotations. This enables the system to consider various annotators’ perspectives, biases, or interpretations, resulting in a fairer learning process.
Participants will be asked to classify tweets and memes (in English and Spanish) according to the following six tasks:
TASK 1 - Sexism Identification in Tweets: The first subtask is a binary classification. The systems have to decide whether or not a given text (tweet) contains sexist expressions or behaviors (i.e., it is sexist itself, describes a sexist situation or criticizes a sexist behavior).
TASK 2 - Source Intention in Tweets: For the tweets that have been classified as sexist, the second task aims to classify each tweet according to the intention of the person who wrote it. We propose a ternary classification task: (i) DIRECT sexist message, (ii) REPORTED sexist message and (iii) JUDGEMENTAL message.
TASK 3 - Sexism Categorization in Tweets: Once a message has been classified as sexist, the third task aims to categorize the message in different types of sexism (according to a categorization proposed by experts and that takes into account the different facets of women that are undermined). In particular, each sexist tweet must be categorized in one or more of the following categories: (i) IDEOLOGICAL AND INEQUALITY, (ii) STEREOTYPING AND DOMINANCE, (iii) OBJECTIFICATION, (iv) SEXUAL VIOLENCE and (v) MISOGYNY AND NON-SEXUAL VIOLENCE.
TASK 4 - Sexism Identification in Memes: This is a binary classification task consisting of deciding whether or not a given meme is sexist.
Task 5: Source Intention in Memes: As in Task 2, this task aims to categorize the sexists memes according to the intention of the author. Due to the characteristics of the memes, the REPORTED label is virtually null, so in this task systems should only classify sexist memes in DIRECT or JUDGEMENTAL.
Task 6: Sexism Categorization in Memes: This task aims to classify sexist memes according to the categorization provided for Task 3: (i) IDEOLOGICAL AND INEQUALITY, (ii) STEREOTYPING AND DOMINANCE, (iii) OBJECTIFICATION, (iv) SEXUAL VIOLENCE and (v) MISOGYNY AND NON-SEXUAL VIOLENCE.
Although we recommend to participate in all subtasks, participants are allowed to participate just in one of them (e.g., subtask 1).
During the training phase, the task organizers will provide to the participants the manually-annotated EXIST 2024 dataset. For the evaluation of the teams, the unlabelled test data will be released.
We encourage participation from both academic institutions and industrial organizations. We invite the participants to register for the lab at CLEF 2024 Labs Registration site (https://clef2024-labs-registration.dei.unipd.it/). You will receive information about how to join the Google Group for the EXIST 2024 shared task.
Important Dates:
* 13 November 2023 - Registration open.
* 4 March 2024 - Training and development sets available.
* 15 April 2024 - Test set available.
* 22 April 2024 - Registration closes.
* 6 May 2024 - Runs submission due to organizers.
* 20 May 2024 - Results notification to participants.
* 3 June 2024 - Submission of Working Notes by participants.
* 19 June 2024 - Notification of acceptance (peer-reviews).
* 3 July 2024 - Camera-ready participant papers due to organizers.
* 9-12 September 2024 - EXIST 2024 at CLEF Conference.
** Note: All deadlines are 11:59PM UTC-12:00 ("anywhere on Earth") **
Organizers:
Laura Plaza, Universidad Nacional de Educación a Distancia (UNED)
Jorge Carrillo-de-Albornoz, Universidad Nacional de Educación a Distancia (UNED)
Enrique Amigó, Universidad Nacional de Educación a Distancia (UNED)
Julio Gonzalo, Universidad Nacional de Educación a Distancia (UNED)
Roser Morante, Universidad Nacional de Educación a Distancia (UNED)
Víctor Ruiz García, Universidad Nacional de Educación a Distancia (UNED)
Damiano Spina, Royal Melbourne Institute of Technology (RMIT)
Paolo Rosso, Universitat Politècnica de València (UPV)
Berta Chulvi, Universitat Politècnica de València (UPV)
Alba Maeso Olmos, Universitat Politècnica de València (UPV)
Contact:
Contact the organizers by writing to: jcalbornoz [at] lsi.uned.es
Website: http://nlp.uned.es/exist2024/