TY - JOUR
T1 - Overview of the CLEF-2023 CheckThat! Lab
T2 - 24th Working Notes of the Conference and Labs of the Evaluation Forum, CLEF-WN 2023
AU - Galassi, Andrea
AU - Ruggeri, Federico
AU - Barrón-Cedeño, Alberto
AU - Alam, Firoj
AU - Caselli, Tommaso
AU - Kutlu, Mucahid
AU - Struß, Julia Maria
AU - Antici, Francesco
AU - Hasanain, Maram
AU - Köhler, Juliane
AU - Korre, Katerina
AU - Leistra, Folkert
AU - Muti, Arianna
AU - Siegel, Melanie
AU - Türkmen, Mehmet Deniz
AU - Wiegand, Michael
AU - Zaghouani, Wajdi
N1 - Publisher Copyright:
© 2023 Copyright for this paper by its authors.
PY - 2023
Y1 - 2023
N2 - We describe the outcome of the 2023 edition of the CheckThat!Lab at CLEF. We focus on subjectivity (Task 2), which has been proposed for the first time. It aims at fostering the technology for the identification of subjective text fragments in news articles. For that, we produced corpora consisting of 9,530 manually-annotated sentences, covering six languages —Arabic, Dutch, English, German, Italian, and Turkish. Task 2 attracted 12 teams, which submitted a total of 40 final runs covering all languages. The most successful approaches addressed the task using state-of-the-art multilingual transformer models, which were fine-tuned on language-specific data. Teams also experimented with a rich set of other neural architectures, including foundation models, zero-shot classifiers, and standard transformers, mainly coupled with data augmentation and multilingual training strategies to address class imbalance. We publicly release all the datasets and evaluation scripts, with the purpose of promoting further research on this topic.
AB - We describe the outcome of the 2023 edition of the CheckThat!Lab at CLEF. We focus on subjectivity (Task 2), which has been proposed for the first time. It aims at fostering the technology for the identification of subjective text fragments in news articles. For that, we produced corpora consisting of 9,530 manually-annotated sentences, covering six languages —Arabic, Dutch, English, German, Italian, and Turkish. Task 2 attracted 12 teams, which submitted a total of 40 final runs covering all languages. The most successful approaches addressed the task using state-of-the-art multilingual transformer models, which were fine-tuned on language-specific data. Teams also experimented with a rich set of other neural architectures, including foundation models, zero-shot classifiers, and standard transformers, mainly coupled with data augmentation and multilingual training strategies to address class imbalance. We publicly release all the datasets and evaluation scripts, with the purpose of promoting further research on this topic.
UR - http://www.scopus.com/inward/record.url?scp=85175626781&partnerID=8YFLogxK
M3 - Conference article
AN - SCOPUS:85175626781
SN - 1613-0073
VL - 3497
SP - 236
EP - 249
JO - CEUR Workshop Proceedings
JF - CEUR Workshop Proceedings
Y2 - 18 September 2023 through 21 September 2023
ER -