8 citations · 9 across the 6 of their papers we have counts for
5 papers · 1 filter
To Each Language Its Tokenizer: Modular Tokenizers for Efficient Multilingual LLMs
Franck Signe, Hippolyte Pilchen, François Yvon +1
Multilingual Large Language Models (LLMs) traditionally rely on a single vocabulary shared by all supported languages, which can lead to uneven compression across them. Moreover, t…
ARC-Encoder: learning compressed text representations for large language models
Hippolyte Pilchen, Edouard Grave, Patrick Pérez
Recent techniques such as retrieval-augmented generation or chain-of-thought reasoning have led to longer contexts and increased inference costs. Context compression techniques can…
Streaming Sequence-to-Sequence Learning with Delayed Streams Modeling
Neil Zeghidour, Eugene Kharitonov, Manu Orsini +6
We introduce Delayed Streams Modeling (DSM), a flexible formulation for streaming, multimodal sequence-to-sequence learning. Sequence-to-sequence generation is often cast in an off…
High-Fidelity Simultaneous Speech-To-Speech Translation
Tom Labiausse, Laurent Mazaré, Edouard Grave +3
We introduce Hibiki, a decoder-only model for simultaneous speech translation. Hibiki leverages a multistream language model to synchronously process source and target speech, and…
Neutral Residues: Revisiting Adapters for Model Extension
Franck Signe Talla, Edouard Grave, Hervé Jégou
We address the problem of extending a pretrained large language model to a new domain that was not seen during training. Standard techniques, such as finetuning or low-rank adaptat…