TY - GEN
T1 - Overview of the CLEF-2024 CheckThat! Lab
T2 - 15th International Conference of the CLEF Association, CLEF 2024
AU - Barrón-Cedeño, Alberto
AU - Alam, Firoj
AU - Struß, Julia Maria
AU - Nakov, Preslav
AU - Chakraborty, Tanmoy
AU - Elsayed, Tamer
AU - Przybyła, Piotr
AU - Caselli, Tommaso
AU - Da San Martino, Giovanni
AU - Haouari, Fatima
AU - Hasanain, Maram
AU - Li, Chengkai
AU - Piskorski, Jakub
AU - Ruggeri, Federico
AU - Song, Xingyi
AU - Suwaileh, Reem
N1 - Publisher Copyright:
© The Author(s), under exclusive license to Springer Nature Switzerland AG 2024.
PY - 2024/9/19
Y1 - 2024/9/19
N2 - We describe the seventh edition of the CheckThat! lab, part of the 2024 Conference and Labs of the Evaluation Forum (CLEF). Previous editions of CheckThat! focused on the main tasks of the information verification pipeline: check-worthiness, identifying previously fact-checked claims, supporting evidence retrieval, and claim verification. In this edition, we introduced some new challenges, offering six tasks in fifteen languages (Arabic, Bulgarian, English, Dutch, French, Georgian, German, Greek, Italian, Polish, Portuguese, Russian, Slovene, Spanish, and code-mixed Hindi-English): Task 1 on estimation of check-worthiness (the only task that has been present in all CheckThat! editions), Task 2 on identification of subjectivity (a follow up of the CheckThat! 2023 edition), Task 3 on identification of the use of persuasion techniques (a follow up of SemEval 2023), Task 4 on detection of hero, villain, and victim from memes (a follow up of CONSTRAINT 2022), Task 5 on rumor verification using evidence from authorities (new task), and Task 6 on robustness of credibility assessment with adversarial examples (new task). These are challenging classification and retrieval problems at the document and at the span level, including multilingual and multimodal settings. This year, CheckThat! was one of the most popular labs at CLEF-2024 in terms of team registrations: 130 teams. More than one-third of them (a total of 46) actually participated.
AB - We describe the seventh edition of the CheckThat! lab, part of the 2024 Conference and Labs of the Evaluation Forum (CLEF). Previous editions of CheckThat! focused on the main tasks of the information verification pipeline: check-worthiness, identifying previously fact-checked claims, supporting evidence retrieval, and claim verification. In this edition, we introduced some new challenges, offering six tasks in fifteen languages (Arabic, Bulgarian, English, Dutch, French, Georgian, German, Greek, Italian, Polish, Portuguese, Russian, Slovene, Spanish, and code-mixed Hindi-English): Task 1 on estimation of check-worthiness (the only task that has been present in all CheckThat! editions), Task 2 on identification of subjectivity (a follow up of the CheckThat! 2023 edition), Task 3 on identification of the use of persuasion techniques (a follow up of SemEval 2023), Task 4 on detection of hero, villain, and victim from memes (a follow up of CONSTRAINT 2022), Task 5 on rumor verification using evidence from authorities (new task), and Task 6 on robustness of credibility assessment with adversarial examples (new task). These are challenging classification and retrieval problems at the document and at the span level, including multilingual and multimodal settings. This year, CheckThat! was one of the most popular labs at CLEF-2024 in terms of team registrations: 130 teams. More than one-third of them (a total of 46) actually participated.
KW - Authority Finding
KW - Check-Worthiness
KW - Credibility Assessment
KW - Fact-Checking
KW - Propaganda
KW - Rumor Verification
KW - Subjectivity
UR - http://www.scopus.com/inward/record.url?scp=85201592880&partnerID=8YFLogxK
U2 - 10.1007/978-3-031-71908-0_2
DO - 10.1007/978-3-031-71908-0_2
M3 - Conference contribution
AN - SCOPUS:85201592880
SN - 9783031719073
T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
SP - 28
EP - 52
BT - Experimental IR Meets Multilinguality, Multimodality, and Interaction - 15th International Conference of the CLEF Association, CLEF 2024, Proceedings
A2 - Goeuriot, Lorraine
A2 - Mulhem, Philippe
A2 - Quénot, Georges
A2 - Schwab, Didier
A2 - Di Nunzio, Giorgio Maria
A2 - Faggioli, Guglielmo
A2 - Ferro, Nicola
A2 - Soulier, Laure
A2 - Galuščáková, Petra
A2 - García Seco de Herrera, Alba
PB - Springer Science and Business Media Deutschland GmbH
Y2 - 9 September 2024 through 12 September 2024
ER -