Overview of the CLEF-2023 CheckThat! Lab Task 3 on Political Bias of News Articles and News Media

Giovanni Da San Martino; Firoj Alam; Maram Hasanain; Rabindra Nath Nandi; Dilshod Azizov; Preslav Nakov

Overview of the CLEF-2023 CheckThat! Lab Task 3 on Political Bias of News Articles and News Media

Giovanni Da San Martino^*, Firoj Alam^*, Maram Hasanain, Rabindra Nath Nandi, Dilshod Azizov, Preslav Nakov

^*Corresponding author for this work

Qatar Computing Research Institute

Research output: Contribution to journal › Conference article › peer-review

Abstract

We provide an overview of task 3 of the CheckThat!lab from the Cross-Language Evaluation Forum (CLEF) 2023, which focuses on predicting the political leaning of English-language news articles and news media outlets. We describe the data collection, the task setup, the evaluation outcomes, and the approaches used by the participating teams. A total of six teams submitted runs for the two subtasks. The top-performing system in Subtask 3A achieved a Mean Absolute Error (MAE) of 0.473, while the best system in Subtask 3B yielded a MAE of 0.549. We make all datasets and evaluation scripts available to the public, aiming to boost further research on this problem.

Original language	English
Pages (from-to)	250-259
Number of pages	10
Journal	CEUR Workshop Proceedings
Volume	3497
Publication status	Published - 2023
Event	24th Working Notes of the Conference and Labs of the Evaluation Forum, CLEF-WN 2023 - Thessaloniki, Greece Duration: 18 Sept 2023 → 21 Sept 2023

Keywords

Political bias
news articles
news media

Cite this

@article{d5f701853f794a1c87bd16675c0400c4,

title = "Overview of the CLEF-2023 CheckThat! Lab Task 3 on Political Bias of News Articles and News Media",

abstract = "We provide an overview of task 3 of the CheckThat!lab from the Cross-Language Evaluation Forum (CLEF) 2023, which focuses on predicting the political leaning of English-language news articles and news media outlets. We describe the data collection, the task setup, the evaluation outcomes, and the approaches used by the participating teams. A total of six teams submitted runs for the two subtasks. The top-performing system in Subtask 3A achieved a Mean Absolute Error (MAE) of 0.473, while the best system in Subtask 3B yielded a MAE of 0.549. We make all datasets and evaluation scripts available to the public, aiming to boost further research on this problem.",

keywords = "Political bias, news articles, news media",

author = "{Da San Martino}, Giovanni and Firoj Alam and Maram Hasanain and Nandi, {Rabindra Nath} and Dilshod Azizov and Preslav Nakov",

note = "Publisher Copyright: {\textcopyright} 2023 Copyright for this paper by its authors.; 24th Working Notes of the Conference and Labs of the Evaluation Forum, CLEF-WN 2023 ; Conference date: 18-09-2023 Through 21-09-2023",

year = "2023",

language = "English",

volume = "3497",

pages = "250--259",

journal = "CEUR Workshop Proceedings",

issn = "1613-0073",

publisher = "CEUR-WS",

}

TY - JOUR

T1 - Overview of the CLEF-2023 CheckThat! Lab Task 3 on Political Bias of News Articles and News Media

AU - Da San Martino, Giovanni

AU - Alam, Firoj

AU - Hasanain, Maram

AU - Nandi, Rabindra Nath

AU - Azizov, Dilshod

AU - Nakov, Preslav

PY - 2023

Y1 - 2023

N2 - We provide an overview of task 3 of the CheckThat!lab from the Cross-Language Evaluation Forum (CLEF) 2023, which focuses on predicting the political leaning of English-language news articles and news media outlets. We describe the data collection, the task setup, the evaluation outcomes, and the approaches used by the participating teams. A total of six teams submitted runs for the two subtasks. The top-performing system in Subtask 3A achieved a Mean Absolute Error (MAE) of 0.473, while the best system in Subtask 3B yielded a MAE of 0.549. We make all datasets and evaluation scripts available to the public, aiming to boost further research on this problem.

AB - We provide an overview of task 3 of the CheckThat!lab from the Cross-Language Evaluation Forum (CLEF) 2023, which focuses on predicting the political leaning of English-language news articles and news media outlets. We describe the data collection, the task setup, the evaluation outcomes, and the approaches used by the participating teams. A total of six teams submitted runs for the two subtasks. The top-performing system in Subtask 3A achieved a Mean Absolute Error (MAE) of 0.473, while the best system in Subtask 3B yielded a MAE of 0.549. We make all datasets and evaluation scripts available to the public, aiming to boost further research on this problem.

KW - Political bias

KW - news articles

KW - news media

UR - http://www.scopus.com/inward/record.url?scp=85175652318&partnerID=8YFLogxK

M3 - Conference article

AN - SCOPUS:85175652318

SN - 1613-0073

VL - 3497

SP - 250

EP - 259

JO - CEUR Workshop Proceedings

JF - CEUR Workshop Proceedings

T2 - 24th Working Notes of the Conference and Labs of the Evaluation Forum, CLEF-WN 2023

Y2 - 18 September 2023 through 21 September 2023

ER -

Overview of the CLEF-2023 CheckThat! Lab Task 3 on Political Bias of News Articles and News Media

Abstract

Keywords

Other files and links

Fingerprint

EX-QNRF-NPRPC-7: The Future of Digital Citizenship in Qatar: a Socio-Technical Approach

Cite this

Overview of the CLEF-2023 CheckThat! Lab Task 3 on Political Bias of News Articles and News Media

Abstract

Keywords

Other files and links

Fingerprint

Projects

EX-QNRF-NPRPC-7: The Future of Digital Citizenship in Qatar: a Socio-Technical Approach

Cite this