CheckThat! at CLEF 2020: Enabling the automatic identification and verification of claims in social media

Alberto Barrón-Cedeño; Tamer Elsayed; Preslav Nakov; Giovanni Da San Martino; Maram Hasanain; Reem Suwaileh; Fatima Haouari

doi:10.1007/978-3-030-45442-5_65

CheckThat! at CLEF 2020: Enabling the automatic identification and verification of claims in social media

Alberto Barrón-Cedeño^*, Tamer Elsayed, Preslav Nakov, Giovanni Da San Martino, Maram Hasanain, Reem Suwaileh, Fatima Haouari

^*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

45 Citations (Scopus)

Abstract

We describe the third edition of the CheckThat! Lab, which is part of the 2020 Cross-Language Evaluation Forum (CLEF). CheckThat! proposes four complementary tasks and a related task from previous lab editions, offered in English, Arabic, and Spanish. Task 1 asks to predict which tweets in a Twitter stream are worth fact-checking. Task 2 asks to determine whether a claim posted in a tweet can be verified using a set of previously fact-checked claims. Task 3 asks to retrieve text snippets from a given set of Web pages that would be useful for verifying a target tweet’s claim. Task 4 asks to predict the veracity of a target tweet’s claim using a set of potentially-relevant Web pages. Finally, the lab offers a fifth task that asks to predict the check-worthiness of the claims made in English political debates and speeches. CheckThat! features a full evaluation framework. The evaluation is carried out using mean average precision or precision at rank k for ranking tasks, and F₁ for classification tasks.

Original language	English
Title of host publication	Advances in Information Retrieval - 42nd European Conference on IR Research, ECIR 2020, Proceedings
Editors	Joemon M. Jose, Emine Yilmaz, João Magalhães, Flávio Martins, Pablo Castells, Nicola Ferro, Mário J. Silva
Publisher	Springer
Pages	499-507
Number of pages	9
ISBN (Print)	9783030454418
DOIs	https://doi.org/10.1007/978-3-030-45442-5_65
Publication status	Published - 2020
Externally published	Yes
Event	42nd European Conference on IR Research, ECIR 2020 - Lisbon, Portugal Duration: 14 Apr 2020 → 17 Apr 2020

Publication series

Name	Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume	12036 LNCS
ISSN (Print)	0302-9743
ISSN (Electronic)	1611-3349

Conference

Conference	42nd European Conference on IR Research, ECIR 2020
Country/Territory	Portugal
City	Lisbon
Period	14/04/20 → 17/04/20

Access to Document

10.1007/978-3-030-45442-5_65

Cite this

Barrón-Cedeño, A., Elsayed, T., Nakov, P., Da San Martino, G., Hasanain, M., Suwaileh, R., & Haouari, F. (2020). CheckThat! at CLEF 2020: Enabling the automatic identification and verification of claims in social media. In J. M. Jose, E. Yilmaz, J. Magalhães, F. Martins, P. Castells, N. Ferro, & M. J. Silva (Eds.), Advances in Information Retrieval - 42nd European Conference on IR Research, ECIR 2020, Proceedings (pp. 499-507). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 12036 LNCS). Springer. https://doi.org/10.1007/978-3-030-45442-5_65

Barrón-Cedeño, Alberto ; Elsayed, Tamer ; Nakov, Preslav et al. / CheckThat! at CLEF 2020 : Enabling the automatic identification and verification of claims in social media. Advances in Information Retrieval - 42nd European Conference on IR Research, ECIR 2020, Proceedings. editor / Joemon M. Jose ; Emine Yilmaz ; João Magalhães ; Flávio Martins ; Pablo Castells ; Nicola Ferro ; Mário J. Silva. Springer, 2020. pp. 499-507 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).

@inproceedings{6a48644dfbdd4fb68587e4c7477a301f,

title = "CheckThat! at CLEF 2020: Enabling the automatic identification and verification of claims in social media",

abstract = "We describe the third edition of the CheckThat! Lab, which is part of the 2020 Cross-Language Evaluation Forum (CLEF). CheckThat! proposes four complementary tasks and a related task from previous lab editions, offered in English, Arabic, and Spanish. Task 1 asks to predict which tweets in a Twitter stream are worth fact-checking. Task 2 asks to determine whether a claim posted in a tweet can be verified using a set of previously fact-checked claims. Task 3 asks to retrieve text snippets from a given set of Web pages that would be useful for verifying a target tweet{\textquoteright}s claim. Task 4 asks to predict the veracity of a target tweet{\textquoteright}s claim using a set of potentially-relevant Web pages. Finally, the lab offers a fifth task that asks to predict the check-worthiness of the claims made in English political debates and speeches. CheckThat! features a full evaluation framework. The evaluation is carried out using mean average precision or precision at rank k for ranking tasks, and F1 for classification tasks.",

author = "Alberto Barr{\'o}n-Cede{\~n}o and Tamer Elsayed and Preslav Nakov and {Da San Martino}, Giovanni and Maram Hasanain and Reem Suwaileh and Fatima Haouari",

note = "Publisher Copyright: {\textcopyright} Springer Nature Switzerland AG 2020.; 42nd European Conference on IR Research, ECIR 2020 ; Conference date: 14-04-2020 Through 17-04-2020",

year = "2020",

doi = "10.1007/978-3-030-45442-5_65",

language = "English",

isbn = "9783030454418",

series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",

publisher = "Springer",

pages = "499--507",

editor = "Jose, {Joemon M.} and Emine Yilmaz and Jo{\~a}o Magalh{\~a}es and Fl{\'a}vio Martins and Pablo Castells and Nicola Ferro and Silva, {M{\'a}rio J.}",

booktitle = "Advances in Information Retrieval - 42nd European Conference on IR Research, ECIR 2020, Proceedings",

address = "Germany",

}

Barrón-Cedeño, A, Elsayed, T, Nakov, P, Da San Martino, G, Hasanain, M, Suwaileh, R & Haouari, F 2020, CheckThat! at CLEF 2020: Enabling the automatic identification and verification of claims in social media. in JM Jose, E Yilmaz, J Magalhães, F Martins, P Castells, N Ferro & MJ Silva (eds), Advances in Information Retrieval - 42nd European Conference on IR Research, ECIR 2020, Proceedings. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 12036 LNCS, Springer, pp. 499-507, 42nd European Conference on IR Research, ECIR 2020, Lisbon, Portugal, 14/04/20. https://doi.org/10.1007/978-3-030-45442-5_65

CheckThat! at CLEF 2020: Enabling the automatic identification and verification of claims in social media. / Barrón-Cedeño, Alberto; Elsayed, Tamer; Nakov, Preslav et al.
Advances in Information Retrieval - 42nd European Conference on IR Research, ECIR 2020, Proceedings. ed. / Joemon M. Jose; Emine Yilmaz; João Magalhães; Flávio Martins; Pablo Castells; Nicola Ferro; Mário J. Silva. Springer, 2020. p. 499-507 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 12036 LNCS).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - CheckThat! at CLEF 2020

T2 - 42nd European Conference on IR Research, ECIR 2020

AU - Barrón-Cedeño, Alberto

AU - Elsayed, Tamer

AU - Nakov, Preslav

AU - Da San Martino, Giovanni

AU - Hasanain, Maram

AU - Suwaileh, Reem

AU - Haouari, Fatima

N1 - Publisher Copyright: © Springer Nature Switzerland AG 2020.

PY - 2020

Y1 - 2020

N2 - We describe the third edition of the CheckThat! Lab, which is part of the 2020 Cross-Language Evaluation Forum (CLEF). CheckThat! proposes four complementary tasks and a related task from previous lab editions, offered in English, Arabic, and Spanish. Task 1 asks to predict which tweets in a Twitter stream are worth fact-checking. Task 2 asks to determine whether a claim posted in a tweet can be verified using a set of previously fact-checked claims. Task 3 asks to retrieve text snippets from a given set of Web pages that would be useful for verifying a target tweet’s claim. Task 4 asks to predict the veracity of a target tweet’s claim using a set of potentially-relevant Web pages. Finally, the lab offers a fifth task that asks to predict the check-worthiness of the claims made in English political debates and speeches. CheckThat! features a full evaluation framework. The evaluation is carried out using mean average precision or precision at rank k for ranking tasks, and F1 for classification tasks.

AB - We describe the third edition of the CheckThat! Lab, which is part of the 2020 Cross-Language Evaluation Forum (CLEF). CheckThat! proposes four complementary tasks and a related task from previous lab editions, offered in English, Arabic, and Spanish. Task 1 asks to predict which tweets in a Twitter stream are worth fact-checking. Task 2 asks to determine whether a claim posted in a tweet can be verified using a set of previously fact-checked claims. Task 3 asks to retrieve text snippets from a given set of Web pages that would be useful for verifying a target tweet’s claim. Task 4 asks to predict the veracity of a target tweet’s claim using a set of potentially-relevant Web pages. Finally, the lab offers a fifth task that asks to predict the check-worthiness of the claims made in English political debates and speeches. CheckThat! features a full evaluation framework. The evaluation is carried out using mean average precision or precision at rank k for ranking tasks, and F1 for classification tasks.

UR - http://www.scopus.com/inward/record.url?scp=85084188926&partnerID=8YFLogxK

U2 - 10.1007/978-3-030-45442-5_65

DO - 10.1007/978-3-030-45442-5_65

M3 - Conference contribution

AN - SCOPUS:85084188926

SN - 9783030454418

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 499

EP - 507

BT - Advances in Information Retrieval - 42nd European Conference on IR Research, ECIR 2020, Proceedings

A2 - Jose, Joemon M.

A2 - Yilmaz, Emine

A2 - Magalhães, João

A2 - Martins, Flávio

A2 - Castells, Pablo

A2 - Ferro, Nicola

A2 - Silva, Mário J.

PB - Springer

Y2 - 14 April 2020 through 17 April 2020

ER -

Barrón-Cedeño A, Elsayed T, Nakov P, Da San Martino G, Hasanain M, Suwaileh R et al. CheckThat! at CLEF 2020: Enabling the automatic identification and verification of claims in social media. In Jose JM, Yilmaz E, Magalhães J, Martins F, Castells P, Ferro N, Silva MJ, editors, Advances in Information Retrieval - 42nd European Conference on IR Research, ECIR 2020, Proceedings. Springer. 2020. p. 499-507. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). doi: 10.1007/978-3-030-45442-5_65

CheckThat! at CLEF 2020: Enabling the automatic identification and verification of claims in social media

Abstract

Publication series

Conference

Access to Document

Other files and links

Fingerprint

Cite this