The CLEF-2023 CheckThat! Lab: Checkworthiness, Subjectivity, Political Bias, Factuality, and Authority

Alberto Barrón-Cedeño; Firoj Alam; Tommaso Caselli; Giovanni Da San Martino; Tamer Elsayed; Andrea Galassi; Fatima Haouari; Federico Ruggeri; Julia Maria Struß; Rabindra Nath Nandi; Gullal S. Cheema; Dilshod Azizov; Preslav Nakov

doi:10.1007/978-3-031-28241-6_59

The CLEF-2023 CheckThat! Lab: Checkworthiness, Subjectivity, Political Bias, Factuality, and Authority

Alberto Barrón-Cedeño^*, Firoj Alam, Tommaso Caselli, Giovanni Da San Martino, Tamer Elsayed, Andrea Galassi, Fatima Haouari, Federico Ruggeri, Julia Maria Struß, Rabindra Nath Nandi, Gullal S. Cheema, Dilshod Azizov, Preslav Nakov

^*Corresponding author for this work

Qatar Computing Research Institute

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

36 Citations (Scopus)

Abstract

The five editions of the CheckThat! lab so far have focused on the main tasks of the information verification pipeline: check-worthiness, evidence retrieval and pairing, and verification. The 2023 edition of the lab zooms into some of the problems and—for the first time—it offers five tasks in seven languages (Arabic, Dutch, English, German, Italian, Spanish, and Turkish): Task 1 asks to determine whether an item, text or a text plus an image, is check-worthy; Task 2 requires to assess whether a text snippet is subjective or not; Task 3 looks for estimating the political bias of a document or a news outlet; Task 4 requires to determine the level of factuality of a document or a news outlet; and Task 5 is about identifying authorities that should be trusted to verify a contended claim.

Original language	English
Title of host publication	Advances in Information Retrieval - 45th European Conference on Information Retrieval, ECIR 2023, Proceedings
Editors	Jaap Kamps, Lorraine Goeuriot, Fabio Crestani, Maria Maistro, Hideo Joho, Brian Davis, Cathal Gurrin, Annalina Caputo, Udo Kruschwitz
Publisher	Springer Science and Business Media Deutschland GmbH
Pages	506-517
Number of pages	12
ISBN (Print)	9783031282409
DOIs	https://doi.org/10.1007/978-3-031-28241-6_59
Publication status	Published - 2023
Event	45th European Conference on Information Retrieval, ECIR 2023 - Dublin, Ireland Duration: 2 Apr 2023 → 6 Apr 2023

Publication series

Name	Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume	13982 LNCS
ISSN (Print)	0302-9743
ISSN (Electronic)	1611-3349

Conference

Conference	45th European Conference on Information Retrieval, ECIR 2023
Country/Territory	Ireland
City	Dublin
Period	2/04/23 → 6/04/23

Keywords

Authority finding
Check-worthiness
Disinformation
Fact-checking
Factuality
Political bias
Subjectivity

Access to Document

10.1007/978-3-031-28241-6_59

Cite this

Barrón-Cedeño, A., Alam, F., Caselli, T., Da San Martino, G., Elsayed, T., Galassi, A., Haouari, F., Ruggeri, F., Struß, J. M., Nandi, R. N., Cheema, G. S., Azizov, D., & Nakov, P. (2023). The CLEF-2023 CheckThat! Lab: Checkworthiness, Subjectivity, Political Bias, Factuality, and Authority. In J. Kamps, L. Goeuriot, F. Crestani, M. Maistro, H. Joho, B. Davis, C. Gurrin, A. Caputo, & U. Kruschwitz (Eds.), Advances in Information Retrieval - 45th European Conference on Information Retrieval, ECIR 2023, Proceedings (pp. 506-517). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 13982 LNCS). Springer Science and Business Media Deutschland GmbH. https://doi.org/10.1007/978-3-031-28241-6_59

Barrón-Cedeño, Alberto ; Alam, Firoj ; Caselli, Tommaso et al. / The CLEF-2023 CheckThat! Lab : Checkworthiness, Subjectivity, Political Bias, Factuality, and Authority. Advances in Information Retrieval - 45th European Conference on Information Retrieval, ECIR 2023, Proceedings. editor / Jaap Kamps ; Lorraine Goeuriot ; Fabio Crestani ; Maria Maistro ; Hideo Joho ; Brian Davis ; Cathal Gurrin ; Annalina Caputo ; Udo Kruschwitz. Springer Science and Business Media Deutschland GmbH, 2023. pp. 506-517 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).

@inproceedings{410ec0d9f06843ec891336dee492fb90,

title = "The CLEF-2023 CheckThat! Lab: Checkworthiness, Subjectivity, Political Bias, Factuality, and Authority",

abstract = "The five editions of the CheckThat! lab so far have focused on the main tasks of the information verification pipeline: check-worthiness, evidence retrieval and pairing, and verification. The 2023 edition of the lab zooms into some of the problems and—for the first time—it offers five tasks in seven languages (Arabic, Dutch, English, German, Italian, Spanish, and Turkish): Task 1 asks to determine whether an item, text or a text plus an image, is check-worthy; Task 2 requires to assess whether a text snippet is subjective or not; Task 3 looks for estimating the political bias of a document or a news outlet; Task 4 requires to determine the level of factuality of a document or a news outlet; and Task 5 is about identifying authorities that should be trusted to verify a contended claim.",

keywords = "Authority finding, Check-worthiness, Disinformation, Fact-checking, Factuality, Political bias, Subjectivity",

author = "Alberto Barr{\'o}n-Cede{\~n}o and Firoj Alam and Tommaso Caselli and {Da San Martino}, Giovanni and Tamer Elsayed and Andrea Galassi and Fatima Haouari and Federico Ruggeri and Stru{\ss}, {Julia Maria} and Nandi, {Rabindra Nath} and Cheema, {Gullal S.} and Dilshod Azizov and Preslav Nakov",

note = "Publisher Copyright: {\textcopyright} 2023, The Author(s), under exclusive license to Springer Nature Switzerland AG.; 45th European Conference on Information Retrieval, ECIR 2023 ; Conference date: 02-04-2023 Through 06-04-2023",

year = "2023",

doi = "10.1007/978-3-031-28241-6_59",

language = "English",

isbn = "9783031282409",

series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",

publisher = "Springer Science and Business Media Deutschland GmbH",

pages = "506--517",

editor = "Jaap Kamps and Lorraine Goeuriot and Fabio Crestani and Maria Maistro and Hideo Joho and Brian Davis and Cathal Gurrin and Annalina Caputo and Udo Kruschwitz",

booktitle = "Advances in Information Retrieval - 45th European Conference on Information Retrieval, ECIR 2023, Proceedings",

address = "Germany",

}

Barrón-Cedeño, A, Alam, F, Caselli, T, Da San Martino, G, Elsayed, T, Galassi, A, Haouari, F, Ruggeri, F, Struß, JM, Nandi, RN, Cheema, GS, Azizov, D & Nakov, P 2023, The CLEF-2023 CheckThat! Lab: Checkworthiness, Subjectivity, Political Bias, Factuality, and Authority. in J Kamps, L Goeuriot, F Crestani, M Maistro, H Joho, B Davis, C Gurrin, A Caputo & U Kruschwitz (eds), Advances in Information Retrieval - 45th European Conference on Information Retrieval, ECIR 2023, Proceedings. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 13982 LNCS, Springer Science and Business Media Deutschland GmbH, pp. 506-517, 45th European Conference on Information Retrieval, ECIR 2023, Dublin, Ireland, 2/04/23. https://doi.org/10.1007/978-3-031-28241-6_59

The CLEF-2023 CheckThat! Lab: Checkworthiness, Subjectivity, Political Bias, Factuality, and Authority. / Barrón-Cedeño, Alberto; Alam, Firoj; Caselli, Tommaso et al.
Advances in Information Retrieval - 45th European Conference on Information Retrieval, ECIR 2023, Proceedings. ed. / Jaap Kamps; Lorraine Goeuriot; Fabio Crestani; Maria Maistro; Hideo Joho; Brian Davis; Cathal Gurrin; Annalina Caputo; Udo Kruschwitz. Springer Science and Business Media Deutschland GmbH, 2023. p. 506-517 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 13982 LNCS).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - The CLEF-2023 CheckThat! Lab

T2 - 45th European Conference on Information Retrieval, ECIR 2023

AU - Barrón-Cedeño, Alberto

AU - Alam, Firoj

AU - Caselli, Tommaso

AU - Da San Martino, Giovanni

AU - Elsayed, Tamer

AU - Galassi, Andrea

AU - Haouari, Fatima

AU - Ruggeri, Federico

AU - Struß, Julia Maria

AU - Nandi, Rabindra Nath

AU - Cheema, Gullal S.

AU - Azizov, Dilshod

AU - Nakov, Preslav

PY - 2023

Y1 - 2023

N2 - The five editions of the CheckThat! lab so far have focused on the main tasks of the information verification pipeline: check-worthiness, evidence retrieval and pairing, and verification. The 2023 edition of the lab zooms into some of the problems and—for the first time—it offers five tasks in seven languages (Arabic, Dutch, English, German, Italian, Spanish, and Turkish): Task 1 asks to determine whether an item, text or a text plus an image, is check-worthy; Task 2 requires to assess whether a text snippet is subjective or not; Task 3 looks for estimating the political bias of a document or a news outlet; Task 4 requires to determine the level of factuality of a document or a news outlet; and Task 5 is about identifying authorities that should be trusted to verify a contended claim.

AB - The five editions of the CheckThat! lab so far have focused on the main tasks of the information verification pipeline: check-worthiness, evidence retrieval and pairing, and verification. The 2023 edition of the lab zooms into some of the problems and—for the first time—it offers five tasks in seven languages (Arabic, Dutch, English, German, Italian, Spanish, and Turkish): Task 1 asks to determine whether an item, text or a text plus an image, is check-worthy; Task 2 requires to assess whether a text snippet is subjective or not; Task 3 looks for estimating the political bias of a document or a news outlet; Task 4 requires to determine the level of factuality of a document or a news outlet; and Task 5 is about identifying authorities that should be trusted to verify a contended claim.

KW - Authority finding

KW - Check-worthiness

KW - Disinformation

KW - Fact-checking

KW - Factuality

KW - Political bias

KW - Subjectivity

UR - http://www.scopus.com/inward/record.url?scp=85151062275&partnerID=8YFLogxK

U2 - 10.1007/978-3-031-28241-6_59

DO - 10.1007/978-3-031-28241-6_59

M3 - Conference contribution

AN - SCOPUS:85151062275

SN - 9783031282409

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 506

EP - 517

BT - Advances in Information Retrieval - 45th European Conference on Information Retrieval, ECIR 2023, Proceedings

A2 - Kamps, Jaap

A2 - Goeuriot, Lorraine

A2 - Crestani, Fabio

A2 - Maistro, Maria

A2 - Joho, Hideo

A2 - Davis, Brian

A2 - Gurrin, Cathal

A2 - Caputo, Annalina

A2 - Kruschwitz, Udo

PB - Springer Science and Business Media Deutschland GmbH

Y2 - 2 April 2023 through 6 April 2023

ER -

Barrón-Cedeño A, Alam F, Caselli T, Da San Martino G, Elsayed T, Galassi A et al. The CLEF-2023 CheckThat! Lab: Checkworthiness, Subjectivity, Political Bias, Factuality, and Authority. In Kamps J, Goeuriot L, Crestani F, Maistro M, Joho H, Davis B, Gurrin C, Caputo A, Kruschwitz U, editors, Advances in Information Retrieval - 45th European Conference on Information Retrieval, ECIR 2023, Proceedings. Springer Science and Business Media Deutschland GmbH. 2023. p. 506-517. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). doi: 10.1007/978-3-031-28241-6_59