1 citations · 2 across the 24 of their papers we have counts for
26 papers
Last Translation Benchmark
Vilém Zouhar, Niyati Bafna, Mukund Choudhary +263
For scientific progress, we need benchmarks that test the limits of state-of-the-art models, and evaluation methods that inform us about failure cases. As models get stronger, stan…
Asymmetric Within-Document Predictive Learning for Scientific Document Representation
You Zuo, Éric de la Clergerie, Benoît Sagot
We study predictive pretraining for scientific document representation using the discourse structure of papers. We propose SciJEPA, a citation-free framework that learns through as…
Sparse Coverage: Semantic Center Representations for Patent Prior-Art Retrieval
You Zuo, Kim Gerdes, Éric de la Clergerie +1
Patent prior-art retrieval is a recall-oriented search task over long and highly structured technical documents. Dense retrieval improves semantic matching, but single-vector repre…
Testing the Deliteralization Hypothesis in Human and Machine Translation
Malik Marmonier, Rachel Bawden, Benoît Sagot
The recent shift from dedicated NMT systems to general-purpose LLMs has reshaped machine translation, with LLMs reported to produce more fluent, less literal output than their pred…
Hindsight Quality Prediction Experiments in Multi-Candidate Human-Post-Edited Machine Translation
Malik Marmonier, Benoît Sagot, Rachel Bawden
This paper investigates two complementary paradigms for predicting machine translation (MT) quality: source-side difficulty prediction and candidate-side quality estimation (QE). T…
Translation Heads: Disentangling meaning from language in LLM-based machine translation
Théo Lasnier, Armel Zebaze, Djamé Seddah +2
Mechanistic Interpretability (MI) seeks to explain how neural networks implement their capabilities, but the scale of Large Language Models (LLMs) has limited prior MI work in Mach…
When the Gold Standard Isn't Necessarily Standard: Challenges of Evaluating the Translation of User-Generated Content
Lydia Nishimwe, Benoît Sagot, Rachel Bawden
User-generated content (UGC) is characterised by frequent use of non-standard language, from spelling errors to expressive choices such as slang, character repetitions, and emojis.…
Gaperon: A Peppered English-French Generative Language Model Suite
Nathan Godey, Wissam Antoun, Rian Touchent +4
We release Gaperon, a fully open suite of French-English-coding language models designed to advance transparency and reproducibility in large-scale model training. The Gaperon fami…
LLM Reasoning for Machine Translation: Synthetic Data Generation over Thinking Tokens
Armel Zebaze, Rachel Bawden, Benoît Sagot
Large reasoning models (LRMs) have led to new possibilities in terms of problem-solving, through the devising of a natural language thought process prior to answering a query. Whil…
TopXGen: Topic-Diverse Parallel Data Generation for Low-Resource Machine Translation
Armel Zebaze, Benoît Sagot, Rachel Bawden
LLMs have been shown to perform well in machine translation (MT) with the use of in-context learning (ICL), rivaling supervised models when translating into high-resource languages…
A French Version of the OLDI Seed Corpus
Malik Marmonier, Benoît Sagot, Rachel Bawden
We present the first French partition of the OLDI Seed Corpus, our submission to the WMT 2025 Open Language Data Initiative (OLDI) shared task. We detail its creation process, whic…
ModernBERT or DeBERTaV3? Examining Architecture and Data Influence on Transformer Encoder Models Performance
Wissam Antoun, Benoît Sagot, Djamé Seddah
Pretrained transformer-encoder models like DeBERTaV3 and ModernBERT introduce architectural advancements aimed at improving efficiency and performance. Although the authors of Mode…
Compositional Translation: A Novel LLM-based Approach for Low-resource Machine Translation
Armel Zebaze, Benoît Sagot, Rachel Bawden
The ability of generative large language models (LLMs) to perform in-context learning has given rise to a large body of research into how best to prompt models for various natural…
Q-Filters: Leveraging QK Geometry for Efficient KV Cache Compression
Nathan Godey, Alessio Devoto, Yu Zhao +4
Autoregressive language models rely on a Key-Value (KV) Cache, which avoids re-computing past hidden states during generation, making it faster. As model sizes and context lengths…
Explicit Learning and the LLM in Machine Translation
Malik Marmonier, Rachel Bawden, Benoît Sagot
This study explores an LLM's ability to learn new languages using explanations found in a grammar book, a process we term "explicit learning." To rigorously assess this ability, we…
CamemBERT 2.0: A Smarter French Language Model Aged to Perfection
Wissam Antoun, Francis Kulumba, Rian Touchent +3
French language models, such as CamemBERT, have been widely adopted across industries for natural language processing (NLP) tasks, with models like CamemBERT seeing over 4 million…
Tree of Problems: Improving structured problem solving with compositionality
Armel Zebaze, Benoît Sagot, Rachel Bawden
Large Language Models (LLMs) have demonstrated remarkable performance across multiple tasks through in-context learning. For complex reasoning tasks that require step-by-step think…
Molyé: A Corpus-based Approach to Language Contact in Colonial France
Rasul Dent, Juliette Janès, Thibault Clérice +2
Whether or not several Creole languages which developed during the early modern period can be considered genetic descendants of European languages has been the subject of intense d…
In-Context Example Selection via Similarity Search Improves Low-Resource Machine Translation
Armel Zebaze, Benoît Sagot, Rachel Bawden
The ability of generative large language models (LLMs) to perform in-context learning has given rise to a large body of research into how best to prompt models for various natural…
Towards Zero-Shot Multimodal Machine Translation
Matthieu Futeral, Cordelia Schmid, Benoît Sagot +1
Current multimodal machine translation (MMT) systems rely on fully supervised data (i.e models are trained on sentences with their translations and accompanying images). However, t…
PatentEval: Understanding Errors in Patent Generation
You Zuo, Kim Gerdes, Eric Villemonte de La Clergerie +1
In this work, we introduce a comprehensive error typology specifically designed for evaluating two distinct tasks in machine-generated patent texts: claims-to-abstract generation,…
mOSCAR: A Large-scale Multilingual and Multimodal Document-level Corpus
Matthieu Futeral, Armel Zebaze, Pedro Ortiz Suarez +5
Multimodal Large Language Models (mLLMs) are trained on a large amount of text-image data. While most mLLMs are trained on caption-like data only, Alayrac et al. (2022) showed that…
Why do small language models underperform? Studying Language Model Saturation via the Softmax Bottleneck
Nathan Godey, Éric de la Clergerie, Benoît Sagot
Recent advances in language modeling consist in pretraining highly parameterized neural networks on extremely large web-mined text corpora. Training and inference with such models…
Making Sentence Embeddings Robust to User-Generated Content
Lydia Nishimwe, Benoît Sagot, Rachel Bawden
NLP models have been known to perform poorly on user-generated content (UGC), mainly because it presents a lot of lexical variations and deviates from the standard texts on which m…