373 citations · 446 across the 24 of their papers we have counts for
9 papers · 2 filters
2M-BELEBELE: Highly Multilingual Speech and American Sign Language Comprehension Dataset
Marta R. Costa-jussà, Bokai Yu, Pierre Andrews +7
We introduce the first highly multilingual speech and American Sign Language (ASL) comprehension dataset by extending BELEBELE. Our dataset covers 74 spoken languages at the inters…
Large Concept Models: Language Modeling in a Sentence Representation Space
LCM team, Loïc Barrault, Paul-Ambroise Duquenne +18
LLMs have revolutionized the field of artificial intelligence and have emerged as the de-facto tool for many tasks. The current established technology of LLMs is to process input a…
Y-NQ: English-Yorùbá Evaluation dataset for Open-Book Reading Comprehension and Text Generation
Marta R. Costa-jussà, Joy Chen, Ifeoluwanimi Adebara +3
The purpose of this work is to share an English-Yorùbá evaluation dataset for open-book reading comprehension and text generation to assess the performance of models both in a high…
LCFO: Long Context and Long Form Output Dataset and Benchmarking
Marta R. Costa-jussà, Pierre Andrews, Mariano Coria Meglioli +10
This paper presents the Long Context and Form Output (LCFO) benchmark, a novel evaluation framework for assessing gradual summarization and summary expansion capabilities across di…
On the Role of Speech Data in Reducing Toxicity Detection Bias
Samuel J. Bell, Mariano Coria Meglioli, Megan Richards +6
Text toxicity detection systems exhibit significant biases, producing disproportionate rates of false positives on samples mentioning demographic groups. But what about toxicity de…
Linguini: A benchmark for language-agnostic linguistic reasoning
Eduardo Sánchez, Belen Alastruey, Christophe Ropers +3
We propose a new benchmark to measure a language model's linguistic reasoning skills without relying on pre-existing language-specific knowledge. The test covers 894 questions grou…
FuxiTranyu: A Multilingual Large Language Model Trained with Balanced Data
Haoran Sun, Renren Jin, Shaoyang Xu +10
Large language models (LLMs) have demonstrated prowess in a wide range of tasks. However, many LLMs exhibit significant performance discrepancies between high- and low-resource lan…
Towards Massive Multilingual Holistic Bias
Xiaoqing Ellen Tan, Prangthip Hansanti, Carleigh Wood +3
In the current landscape of automatic language generation, there is a need to understand, evaluate, and mitigate demographic biases as existing models are becoming increasingly mul…
Towards Red Teaming in Multimodal and Multilingual Translation
Christophe Ropers, David Dale, Prangthip Hansanti +9
Assessing performance in Natural Language Processing is becoming increasingly complex. One particular challenge is the potential for evaluation datasets to overlap with training da…