TY - GEN
T1 - SemEval-2022 Task 3
T2 - 16th International Workshop on Semantic Evaluation, SemEval 2022
AU - Zamparelli, Roberto
AU - Chowdhury, Shammur A.
AU - Brunato, Dominique
AU - Chesi, Cristiano
AU - Dell'Orletta, Felice
AU - Hasan, Arid
AU - Venturi, Giulia
N1 - Publisher Copyright:
© 2022 Association for Computational Linguistics.
PY - 2022
Y1 - 2022
N2 - We report the results of the SemEval 2022 Task 3, PreTENS, on evaluation the acceptability of simple sentences containing constructions whose two arguments are presupposed to be or not to be in an ordered taxonomic relation. The task featured two sub-tasks articulated as: (i) binary prediction task and (ii) regression task, predicting the acceptability in a continuous scale. The sentences were artificially generated in three languages (English, Italian and French). 21 systems, with 8 system papers were submitted for the task, all based on various types of fine-tuned transformer systems, often with ensemble methods and various data augmentation techniques. The best systems reached an F1-macro score of 94.49 (sub-task1) and a Spearman correlation coefficient of 0.80 (sub-task2), with interesting variations in specific constructions and/or languages.
AB - We report the results of the SemEval 2022 Task 3, PreTENS, on evaluation the acceptability of simple sentences containing constructions whose two arguments are presupposed to be or not to be in an ordered taxonomic relation. The task featured two sub-tasks articulated as: (i) binary prediction task and (ii) regression task, predicting the acceptability in a continuous scale. The sentences were artificially generated in three languages (English, Italian and French). 21 systems, with 8 system papers were submitted for the task, all based on various types of fine-tuned transformer systems, often with ensemble methods and various data augmentation techniques. The best systems reached an F1-macro score of 94.49 (sub-task1) and a Spearman correlation coefficient of 0.80 (sub-task2), with interesting variations in specific constructions and/or languages.
UR - http://www.scopus.com/inward/record.url?scp=85137606668&partnerID=8YFLogxK
M3 - Conference contribution
AN - SCOPUS:85137606668
T3 - SemEval 2022 - 16th International Workshop on Semantic Evaluation, Proceedings of the Workshop
SP - 228
EP - 238
BT - SemEval 2022 - 16th International Workshop on Semantic Evaluation, Proceedings of the Workshop
A2 - Emerson, Guy
A2 - Schluter, Natalie
A2 - Stanovsky, Gabriel
A2 - Kumar, Ritesh
A2 - Palmer, Alexis
A2 - Schneider, Nathan
A2 - Singh, Siddharth
A2 - Ratan, Shyam
PB - Association for Computational Linguistics (ACL)
Y2 - 14 July 2022 through 15 July 2022
ER -