CALL FOR NTCIR-16 TASK PARTICIPATION
NTCIR-16 tasks: http://research.nii.ac.jp/ntcir/ntcir-16/tasks.html
NTCIR-16 registration: http://research.nii.ac.jp/ntcir/ntcir-16/howto.html
Registration Due:
- DataSearch-2 Aug 31, 2021 (IR Subtasks), Dec 20, 2021 (QA and UI Subtasks)
- DialEval-2 Dec 1, 2021
- FinNum-3 Jan 3, 2022
- Lifelog-4 Dec 1, 2021
- QA Lab-PoliInfo-3 Dec 1, 2021
- WWW-4 Oct 1, 2021
- RCIR Dec 1, 2021
- Real-MedNLP Sep 1, 2021
- SS Aug 1, 2021
- ULTRE Aug 1, 2021
____________________________________________________________
NTCIR-16 now calls for task participation of anyone interested in research on information access technologies and their evaluation, such as retrieval from a large amount of document collections, question answering and natural language processing. We welcome students, young researchers, professors who supervise students, researchers working for a company, and anyone who is interested in informatics.
== What is NTCIR? ==
Development of Information Access technologies based on techniques of Information Retrieval, Natural Language Processing, and Database Management becomes increasingly more important for many applications, (e.g., providing effective access to Web resources and text archives, and analyzing big data obtained from various kinds of sensors). It is indispensable for developing such technologies to experimentally evaluate them by using test collections constructed under collaborations of many researchers. Over the past 20 years, NTCIR has been formulating the infrastructure for the evaluation, and contributing to development of the Information Access technologies. A total of over 80 "evaluation tasks" have been organized, attracting over 1,000 participant research groups worldwide so far. Furthermore, over 4,600 research groups have signed up to use the NTCIR test collections in their research. Consequently, NTCIR has been a major forum for researchers to intensively discuss the evaluation methodology of emerging information access technologies.
== NTCIR-16 Tasks ==
NTCIR-16 Program Committee has selected 10 tasks.
The overview slide of each task can be available at the
kick-off page: http://research.nii.ac.jp/ntcir/ntcir-16/kickoff.html
For more details, please visit websites of each task:
Task Overview page: http://research.nii.ac.jp/ntcir/ntcir-16/tasks.html
NTCIR-16 Tasks
----------------------------------------------------------------------------
CORE TASKS
Data Search 2: Data Search 2
"Search and question answering for statistical data"
Website: https://ntcir.datasearch.jp/
DialEval-2: Dialogue Evaluation 2
"Estimating the quality of a customer-helpdesk dialogue; understanding the role of each dialogue turn towards problem solving"
Website: http://sakailab.com/dialeval2/
FinNum-3: Investor’s and Manager’s Fine-grained Claim Detection
"Investor’s and Manager’s Fine-grained Claim Detection"
Website: https://sites.google.com/nlg.csie.ntu.edu.tw/finnum3/
Lifelog-4: Lifelog Access and Retrieval
"Evalutating approaches to information retrieval from multimodal lifelogs"
Website: http://ntcir-lifelog.computing.dcu.ie/
QA Lab-PoliInfo-3: Question Answering Lab for Political Information
"QA Lab-PoliInfo-3 aims to solve four tasks Question Answering, QA Alignment, Fact Verification and Budget Argument Mining) in political issues."
Website: https://poliinfo3.net
WWW-4: We Want Web 4 with CENTRE
"Quantifying advances and reproducibility in web search"
Website: http://sakailab.com/www4/
PILOT TASKS
RCIR: Reading Comprehension for Information Retrieval
"Text reading signals in Information Retrieval"
Website: http://ntcir-rcir.computing.dcu.ie/
Real-MedNLP: Real document-based Medical Natural Language Processing
"Medical natural language processing using real medical documents"
Website: https://sociocom.naist.jp/real-mednlp/
SS: Session Search
"Session search tasks based on practical data"
http://www.thuir.cn/session-search/
ULTRE: Unbiased Learning to Ranking Evaluation Task
"Evaluating unbiased learning-to-rank with user simulation"
http://ultre.online/
== How to Participate ==
1. Please carefully read "How to Participate to NTCIR-16 Task(s)":
http://research.nii.ac.jp/ntcir/ntcir-16/howto.html
2. Please register at http://ntcir.nii.ac.jp/index.php/ntcir-16-registration-form/
== Schedule ==
(*Schedule can be different in different tasks. Please visit
webpages of each task for the details.)
*Jul, 2021: Dataset Release
*Jul-Sep, 2021: Dry Run
*Sep-Dec, 2021: Formal Run
Feb 1, 2022: Evaluation Result Release to the participants
Feb 1, 2022: Task overview paper release (draft)
Mar 1, 2022: Submission due of participant papers
May 1, 2022: Camera-ready participant paper due
Jun, 2022: NTCIR-16 Conference (NII, Tokyo, Japan)
(At the NTCIR-16 Conference, online presentation will be available)
== Questions? ==
For information regarding the task specifications etc.,
please contact the NTCIR Program Chairs: ntc16-pcc@nii.ac.jp or task organizers.
For information regarding the online registration and previous NTCIR
test collections,
please contact the NTCIR office: ntc-secretariat@nii.ac.jp
We are looking forward to your participation!
NTCIR-16 Program Co-Chairs:
Takehiro Yamamoto (Co-chair, University of Hyogo, Japan)
Zhicheng Dou (Co-chair, Renmin University of China)
NTCIR-16 General Co-Chairs:
Charles Clarke (University of Waterloo, Canada)
Noriko Kando (National Institute of Informatics, Japan)
Makoto P. Kato (University of Tsukuba, Japan)
Yiqun Liu (Tsinghua University, China)