4 papers
Last Translation Benchmark
Vilém Zouhar, Niyati Bafna, Mukund Choudhary +263
For scientific progress, we need benchmarks that test the limits of state-of-the-art models, and evaluation methods that inform us about failure cases. As models get stronger, stan…
MigrationNarrate: A Dataset for Detection of Migration Narratives in YouTube Videos
Fatima Haouari, Carolina Scarton, Kalina Bontcheva
Narratives are central to how social communication is framed, making their detection critical for understanding and analysing public discourse. Prior work has explored narrative de…
Global PIQA: Evaluating Commonsense Reasoning Across 100+ Languages and Cultures
Tyler A. Chang, Catherine Arnett, Abdelrahman Sadallah +377
To date, there exist almost no culturally-specific evaluation benchmarks for large language models (LLMs) that cover a large number of languages and cultures. In this paper, we pre…
UKElectionNarratives: A Dataset of Misleading Narratives Surrounding Recent UK General Elections
Fatima Haouari, Carolina Scarton, Nicolò Faggiani +5
Misleading narratives play a crucial role in shaping public opinion during elections, as they can influence how voters perceive candidates and political parties. This entails the n…