The Regulatory Information Retrieval and Answer Generation (RIRAG) Shared Task will take place as part of the RegNLP 2025 Workshop on January 20th, 2025, in conjunction with the COLING 2025 conference in Abu Dhabi, UAE.
Regulatory documents contain critical information that governs compliance and legal obligations across industries. However, retrieving relevant information and generating precise answers from these documents remains a challenging task due to their complex and diverse nature. The RIRAG Shared Task aims to address this challenge by encouraging the development of state-of-the-art systems for the retrieval and generation of answers from regulatory documents.
===Shared Task Objectives===
-Task 1: Regulatory Information Retrieval
Participants will build models that retrieve relevant passages from a collection of regulatory documents, given a set of questions.
-Task 2: Regulatory Answer Generation
Participants will generate concise and accurate answers to regulatory questions based on the retrieved passages.
===Participation Details===
Participants will be provided with a comprehensive dataset, including regulatory documents and a set of questions related to compliance and obligations.
The shared task will be divided into three phases: the Development Phase, the Testing and Submission Phase, and the Evaluation Phase. Participants will also be invited to describe their system in a paper for the RegNLP workshop proceedings.
===Important Dates===
-Development Phase:
--Start: September 10, 2024
--End: November 10, 2024
Description: Participants can use all provided data for both Task 1 and Task 2. During this phase, test data will be made available for participants to evaluate their own models, but results will not determine final awards.
-Testing and Submission Phase:
--Start: November 10, 2024
--End: November 20, 2024
Description: Unseen questions will be released for participants to apply their models. Final submissions must be made during this period.
-Evaluation Phase:
--Start: November 20, 2024
--End: November 25, 2024
Description: Organizers will evaluate the submitted results, and winners will be determined based on their performance.
-System Paper Submission Phase:
Start: November 25, 2024
End: December 10, 2024
Description: Winning teams are expected to submit a detailed system paper, outlining their methodologies and findings.
===Join the Shared Task===
Interested participants can join the shared task and access more information by visiting our competition page on Codabench: https://www.codabench.org/competitions/3527/
===Baseline System===
For reference, participants can access the baseline system : https://arxiv.org/abs/2409.05677
===More Information===
For more information about RegNLP and RIRAG, please visit our website: https://regnlp.github.io/
===Contact Information===
For inquiries related to the shared task, please contact us via email at: regnlp2025 [at] gmail.com