EveTAR: A new test collection for event detection in Arabic tweets

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

11 Citations (Scopus)

Abstract

Research on event detection in Twitter is often obstructed by the lack of publicly-available evaluation mechanisms such as test collections; this problem is more severe when considering the scarcity of them in languages other than English. In this paper, we present EveTAR, the first publicly-available test collection for event detection in Arabic tweets. The collection includes a crawl of 590M Arabic tweets posted in a month period and covers 66 significant events (in 8 different categories) for which more than 134k relevance judgments were gathered using crowdsourcing with high average inter-annotator agreement (Kappa value of 0.6). We demonstrate the usability of the collection by evaluating 3 state-of-the-art event detection algorithms. The collection is also designed to support other retrieval tasks, as we show in our experiments with ad-hoc search systems.

Original languageEnglish
Title of host publicationSIGIR 2016 - Proceedings of the 39th International ACM SIGIR Conference on Research and Development in Information Retrieval
PublisherAssociation for Computing Machinery, Inc
Pages689-692
Number of pages4
ISBN (Electronic)9781450342902
DOIs
Publication statusPublished - 7 Jul 2016
Externally publishedYes
Event39th International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR 2016 - Pisa, Italy
Duration: 17 Jul 201621 Jul 2016

Publication series

NameSIGIR 2016 - Proceedings of the 39th International ACM SIGIR Conference on Research and Development in Information Retrieval

Conference

Conference39th International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR 2016
Country/TerritoryItaly
CityPisa
Period17/07/1621/07/16

Keywords

  • Ad-hoc search
  • Crowdsourcing
  • Evaluation
  • Twitter

Fingerprint

Dive into the research topics of 'EveTAR: A new test collection for event detection in Arabic tweets'. Together they form a unique fingerprint.

Cite this