TREC Health Misinformation Track 2020

Call for participation: TREC Health Misinformation Track 2020

https://trec-health-misinfo.github.io

This year, we have focused the track specifically on misinformation related to COVID-19 and SARS-CoV-2, adopting a news corpus from January to April, 2020 as the basis for our test collection. As our understanding of the disease evolved over this period some facts became better known. For example, at one point, it was suggested that Ibuprofen might worsen COVID-19. A retrieval effort undertaken today should avoid returning these articles, or else label them as potentially misleading.

To participate in the track, you need to register by May 31 2020 at the following link:

https://trec.nist.gov/pubs/call2020.html

AIMS AND SCOPE
--------------------------
Misinformation represents a key problem when using search engines to guide any decision-making task: Are users able to discern authoritative from unreliable information and correct from incorrect information? This problem is further exacerbated when the search occurs within uncontrolled data collections, such as the web, where information can be unreliable, misleading, highly technical, and can lead to unfounded escalations. Information from search-engine results can significantly influence decisions, and research shows that increasing the amount of incorrect information about a topic presented in a Search Engine Result Page (SERP) can impel users to make incorrect decisions.

In this context, the TREC 2020 Misinformation track fosters research on retrieval methods that promote reliable and correct information over misinformation. The track offers the following tasks:

* Total Recall Task: The goal is to identify all the documents conveying incorrect information for a specific set of topics;
* Ad-hoc Retrieval Task: The goal is to design a ranking model that promotes credible and correct information over incorrect information;
* Evaluation Meta Task: The goal is to develop new evaluation methods that reflect the credibility and correctness of documents, as well as traditional relevance.

GUIDELINES
------------------
* Corpus: CommonCrawl News crawl from January, 1st 2020 to April 30th, 2020 (https://commoncrawl.org/2016/10/news-dataset-available/);
* Topics: on COVID-19 within the consumer health search domain (people seeking health advice online);
* Runs: runs may be either automatic or manual with the the standard TREC run format.

Detailed guidelines on: https://trec-health-misinfo.github.io

IMPORTANT DATES
-----------------------------
* Registration closes: May 31, 2020
* Runs due from participants: August 2020
* Evaluation results returned: October 2020
* Notebook paper due: October 2020
* TREC 2020 Conference: November 18-20, 2020
* Final paper due: February 2021

ORGANIZERS
--------------------
* Charles Clarke, University of Waterloo
* Maria Maistro, University of Copenhagen
* Mark Smucker, University of Waterloo
* Guido Zuccon, University of Queensland

CONTACT
--------------------
For more information or to ask questions, join the google group: https://groups.google.com/forum/#!forum/trec-decision-track