TY - JOUR
T1 - Overview of the CLEF-2024 CheckThat! Lab Task 1 on Check-Worthiness Estimation of Multigenre Content
AU - Hasanain, Maram
AU - Suwaileh, Reem
AU - Weering, Sanne
AU - Li, Chengkai
AU - Caselli, Tommaso
AU - Zaghouani, Wajdi
AU - Barrón-Cedeño, Alberto
AU - Nakov, Preslav
AU - Alam, Firoj
N1 - Publisher Copyright:
© 2024 Copyright for this paper by its authors.
PY - 2024/9/12
Y1 - 2024/9/12
N2 - We present an overview of the CheckThat! Lab 2024 Task 1, part of CLEF 2024. Task 1 involves determining whether a text item is check-worthy, with a special emphasis on COVID-19, political news, and political debates and speeches. It is conducted in three languages: Arabic, Dutch, and English. Additionally, Spanish was offered for extra training data during the development phase. A total of 75 teams registered, with 37 teams submitting 236 runs and 17 teams submitting system description papers. Out of these, 13, 15 and 26 teams participated for Arabic, Dutch and English, respectively. Among these teams, the use of transformer pre-trained language models (PLMs) was the most frequent. A few teams also employed Large Language Models (LLMs). We provide a description of the dataset, the task setup, including evaluation settings, and a brief overview of the participating systems. As is customary in the CheckThat! Lab, we release all the datasets as well as the evaluation scripts to the research community. This will enable further research on identifying relevant check-worthy content that can assist various stakeholders, such as fact-checkers, journalists, and policymakers.
AB - We present an overview of the CheckThat! Lab 2024 Task 1, part of CLEF 2024. Task 1 involves determining whether a text item is check-worthy, with a special emphasis on COVID-19, political news, and political debates and speeches. It is conducted in three languages: Arabic, Dutch, and English. Additionally, Spanish was offered for extra training data during the development phase. A total of 75 teams registered, with 37 teams submitting 236 runs and 17 teams submitting system description papers. Out of these, 13, 15 and 26 teams participated for Arabic, Dutch and English, respectively. Among these teams, the use of transformer pre-trained language models (PLMs) was the most frequent. A few teams also employed Large Language Models (LLMs). We provide a description of the dataset, the task setup, including evaluation settings, and a brief overview of the participating systems. As is customary in the CheckThat! Lab, we release all the datasets as well as the evaluation scripts to the research community. This will enable further research on identifying relevant check-worthy content that can assist various stakeholders, such as fact-checkers, journalists, and policymakers.
KW - Check-worthiness
KW - fact-checking
KW - multilinguality
UR - http://www.scopus.com/inward/record.url?scp=85201621119&partnerID=8YFLogxK
M3 - Conference article
AN - SCOPUS:85201621119
SN - 1613-0073
VL - 3740
SP - 276
EP - 286
JO - CEUR Workshop Proceedings
JF - CEUR Workshop Proceedings
T2 - 25th Working Notes of the Conference and Labs of the Evaluation Forum, CLEF 2024
Y2 - 9 September 2024 through 12 September 2024
ER -