TY - JOUR
T1 - Overview of CheckThat! 2020 Arabic
T2 - 11th Conference and Labs of the Evaluation Forum, CLEF 2020
AU - Hasanain, Maram
AU - Haouari, Fatima
AU - Suwaileh, Reem
AU - Ali, Zien Sheikh
AU - Hamdan, Bayan
AU - Elsayed, Tamer
AU - Barrón-Cedeño, Alberto
AU - da San Martino, Giovanni
AU - Nakov, Preslav
N1 - Publisher Copyright:
Copyright © 2020 for this paper by its authors.
PY - 2020
Y1 - 2020
N2 - In this paper, we present an overview of the Arabic tasks of the third edition of the CheckThat! Lab at CLEF 2020. The lab featured three Arabic tasks over social media (and the Web): Task 1 on check-worthiness estimation, Task 3 on evidence retrieval, and Task 4 on claim verification. For evaluation, we collected a dataset of Arabic tweets and Web pages consisting of 7.5K tweets and 14,742 Web pages. The systems in the ranking tasks (Task 1 and Task 3) were evaluated using precision at 30 (P@30) and precision at 10 (P@10), respectively. F1 was the official evaluation measure for Task 4. Eight teams submitted runs to the Arabic tasks, which is double the number of teams participating in the Arabic tasks of the CheckThat! lab at CLEF 2019. The most successful approach to Task 1 used an Arabic pre-trained language model, while text similarity measures and linguistic features were used in the other tasks. We release to the research community all datasets from the lab, which should enable further research on automatic claim verification in Arabic social media.
AB - In this paper, we present an overview of the Arabic tasks of the third edition of the CheckThat! Lab at CLEF 2020. The lab featured three Arabic tasks over social media (and the Web): Task 1 on check-worthiness estimation, Task 3 on evidence retrieval, and Task 4 on claim verification. For evaluation, we collected a dataset of Arabic tweets and Web pages consisting of 7.5K tweets and 14,742 Web pages. The systems in the ranking tasks (Task 1 and Task 3) were evaluated using precision at 30 (P@30) and precision at 10 (P@10), respectively. F1 was the official evaluation measure for Task 4. Eight teams submitted runs to the Arabic tasks, which is double the number of teams participating in the Arabic tasks of the CheckThat! lab at CLEF 2019. The most successful approach to Task 1 used an Arabic pre-trained language model, while text similarity measures and linguistic features were used in the other tasks. We release to the research community all datasets from the lab, which should enable further research on automatic claim verification in Arabic social media.
UR - http://www.scopus.com/inward/record.url?scp=85113495519&partnerID=8YFLogxK
M3 - Conference article
AN - SCOPUS:85113495519
SN - 1613-0073
VL - 2696
JO - CEUR Workshop Proceedings
JF - CEUR Workshop Proceedings
Y2 - 22 September 2020 through 25 September 2020
ER -