TY - GEN
T1 - SemEval-2017 Task 3
T2 - 11th International Workshop on Semantic Evaluations, SemEval 2017, co-located with the 55th Annual Meeting of the Association for Computational Linguistics, ACL 2017
AU - Nakov, Preslav
AU - Hoogeveen, Doris
AU - Màrquez, Lluís
AU - Moschitti, Alessandro
AU - Mubarak, Hamdy
AU - Baldwin, Timothy
AU - Verspoor, Karin
N1 - Publisher Copyright:
© 2017 Association for Computational Linguistics
PY - 2017
Y1 - 2017
N2 - We describe SemEval2017 Task 3 on Community Question Answering. This year, we reran the four subtasks from SemEval-2016: (A) Question-Comment Similarity, (B) Question-Question Similarity, (C) Question-External Comment Similarity, and (D) Rerank the correct answers for a new question in Arabic, providing all the data from 2015 and 2016 for training, and fresh data for testing. Additionally, we added a new subtask E in order to enable experimentation with Multi-domain Question Duplicate Detection in a larger-scale scenario, using StackExchange subforums. A total of 23 teams participated in the task, and submitted a total of 85 runs (36 primary and 49 contrastive) for subtasks A-D. Unfortunately, no teams participated in subtask E. A variety of approaches and features were used by the participating systems to address the different subtasks. The best systems achieved an official score (MAP) of 88.43, 47.22, 15.46, and 61.16 in subtasks A, B, C, and D, respectively. These scores are better than the baselines, especially for subtasks A-C.
AB - We describe SemEval2017 Task 3 on Community Question Answering. This year, we reran the four subtasks from SemEval-2016: (A) Question-Comment Similarity, (B) Question-Question Similarity, (C) Question-External Comment Similarity, and (D) Rerank the correct answers for a new question in Arabic, providing all the data from 2015 and 2016 for training, and fresh data for testing. Additionally, we added a new subtask E in order to enable experimentation with Multi-domain Question Duplicate Detection in a larger-scale scenario, using StackExchange subforums. A total of 23 teams participated in the task, and submitted a total of 85 runs (36 primary and 49 contrastive) for subtasks A-D. Unfortunately, no teams participated in subtask E. A variety of approaches and features were used by the participating systems to address the different subtasks. The best systems achieved an official score (MAP) of 88.43, 47.22, 15.46, and 61.16 in subtasks A, B, C, and D, respectively. These scores are better than the baselines, especially for subtasks A-C.
UR - http://www.scopus.com/inward/record.url?scp=85122563846&partnerID=8YFLogxK
M3 - Conference contribution
AN - SCOPUS:85122563846
T3 - Proceedings of the Annual Meeting of the Association for Computational Linguistics
SP - 27
EP - 48
BT - ACL 2017 - 11th International Workshop on Semantic Evaluations, SemEval 2017, Proceedings of the Workshop
PB - Association for Computational Linguistics (ACL)
Y2 - 3 August 2017 through 4 August 2017
ER -