10 citations · 12 across the 6 of their papers we have counts for
6 papers
Are We Evaluating Knowledge or Phrasing? Mitigating MCQA Sensitivity with ParaEval
João Maria Janeiro, Mathurin Videau, Andrea Caciolai +3
Multiple-choice (MCQA) benchmarks are the standard for evaluating pretrained large language models, but their reliance on log-likelihood scoring makes them unreliable. Specifically…
Omnilingual MT: Machine Translation for 1,600 Languages
Omnilingual MT Team, Belen Alastruey, Niyati Bafna +29
High-quality machine translation (MT) can scale to hundreds of languages, setting a high bar for multilingual systems. However, compared to the world's 7,000 languages, current sys…
Interference Matrix: Quantifying Cross-Lingual Interference in Transformer Encoders
Belen Alastruey, João Maria Janeiro, Alexandre Allauzen +3
In this paper, we present a comprehensive study of language interference in encoder-only Transformer models across 83 languages. We construct an interference matrix by training and…
Large Concept Models: Language Modeling in a Sentence Representation Space
LCM team, Loïc Barrault, Paul-Ambroise Duquenne +18
LLMs have revolutionized the field of artificial intelligence and have emerged as the de-facto tool for many tasks. The current established technology of LLMs is to process input a…
MEXMA: Token-level objectives improve sentence representations
João Maria Janeiro, Benjamin Piwowarski, Patrick Gallinari +1
Current pre-trained cross-lingual sentence encoders approaches use sentence-level objectives only. This can lead to loss of information, especially for tokens, which then degrades…
Are Visual Recognition Models Robust to Image Compression?
João Maria Janeiro, Stanislav Frolov, Alaaeldin El-Nouby +1
Reducing the data footprint of visual content via image compression is essential to reduce storage requirements, but also to reduce the bandwidth and latency requirements for trans…