68 citations · 241 across the 19 of their papers we have counts for
21 papers
Large Concept Models: Language Modeling in a Sentence Representation Space
LCM team, Loïc Barrault, Paul-Ambroise Duquenne +18
LLMs have revolutionized the field of artificial intelligence and have emerged as the de-facto tool for many tasks. The current established technology of LLMs is to process input a…
Video Seal: Open and Efficient Video Watermarking
Pierre Fernandez, Hady Elsahar, I. Zeki Yalniz +1
The proliferation of AI-generated content and sophisticated video editing tools has made it both important and challenging to moderate digital platforms. Video watermarking address…
Proactive Detection of Voice Cloning with Localized Watermarking
Robin San Roman, Pierre Fernandez, Alexandre Défossez +3
In the rapidly evolving field of speech generative models, there is a pressing need to ensure audio authenticity against the risks of voice cloning. We present AudioSeal, the first…
Seamless: Multilingual Expressive and Streaming Speech Translation
Seamless Communication, Loïc Barrault, Yu-An Chung +62
Large-scale automatic speech translation systems today lack key features that help machine-mediated communication feel seamless when compared to human-to-human dialogue. In this wo…
SeamlessM4T: Massively Multilingual & Multimodal Machine Translation
Seamless Communication, Loïc Barrault, Yu-An Chung +65
What does it take to create the Babel Fish, a tool that can help individuals translate speech between any two languages? While recent breakthroughs in text-based models have pushed…
What Language Model to Train if You Have One Million GPU Hours?
Teven Le Scao, Thomas Wang, Daniel Hesslow +16
The crystallization of modeling methods around the Transformer architecture has been a boon for practitioners. Simple, well-motivated architectural variations can transfer across t…
BLOOM: A 176B-Parameter Open-Access Multilingual Language Model
BigScience Workshop, :, Teven Le Scao +391
Large language models (LLMs) have been shown to be able to perform new tasks based on a few demonstrations or natural language instructions. While these capabilities have led to wi…
On Reinforcement Learning and Distribution Matching for Fine-Tuning Language Models with no Catastrophic Forgetting
Tomasz Korbak, Hady Elsahar, Germán Kruszewski +1
The availability of large pre-trained models is changing the landscape of Machine Learning research and practice, moving from a training-from-scratch to a fine-tuning paradigm. Whi…
Documenting Geographically and Contextually Diverse Data Sources: The BigScience Catalogue of Language Data and Resources
Angelina McMillan-Major, Zaid Alyafeai, Stella Biderman +15
In recent years, large-scale data collection efforts have prioritized the amount of data collected in order to improve the modeling capabilities of large language models. This prio…
Sampling from Discrete Energy-Based Models with Quality/Efficiency Trade-offs
Bryan Eikema, Germán Kruszewski, Hady Elsahar +1
Energy-Based Models (EBMs) allow for extremely flexible specifications of probability distributions. However, they do not provide a mechanism for obtaining exact samples from these…
Controlling Conditional Language Models without Catastrophic Forgetting
Tomasz Korbak, Hady Elsahar, German Kruszewski +1
Machine learning is shifting towards general-purpose pretrained generative models, trained in a self-supervised manner on large amounts of data, which can then be applied to solve…
Unsupervised and Distributional Detection of Machine-Generated Text
Matthias Gallé, Jos Rozen, Germán Kruszewski +1
The power of natural language generation models has provoked a flurry of interest in automatic methods to detect if a piece of text is human or machine-authored. The problem so far…
Energy-Based Models for Code Generation under Compilability Constraints
Tomasz Korbak, Hady Elsahar, Marc Dymetman +1
Neural language models can be successfully trained on source code, leading to applications such as code completion. However, their versatile autoregressive self-supervision objecti…
References in Wikipedia: The Editors' Perspective
Lucie-Aimée Kaffee, Hady Elsahar
References are an essential part of Wikipedia. Each statement in Wikipedia should be referenced. In this paper, we explore the creation and collection of references for new Wikiped…
A Distributional Approach to Controlled Text Generation
Muhammad Khalifa, Hady Elsahar, Marc Dymetman
We propose a Distributional Approach for addressing Controlled Text Generation from pre-trained Language Models (LMs). This approach permits to specify, in a single formal framewor…
Participatory Research for Low-resourced Machine Translation: A Case Study in African Languages
Wilhelmina Nekoto, Vukosi Marivate, Tshinondiwa Matsila +45
Research in NLP lacks geographic diversity, and the question of how NLP can be scaled to low-resourced languages has not yet been adequately solved. "Low-resourced"-ness is a compl…
Self-Supervised and Controlled Multi-Document Opinion Summarization
Hady Elsahar, Maximin Coavoux, Matthias Gallé +1
We address the problem of unsupervised abstractive summarization of collections of user generated reviews with self-supervision and control. We propose a self-supervised setup that…
Learning to Generate Wikipedia Summaries for Underserved Languages from Wikidata
Lucie-Aimée Kaffee, Hady Elsahar, Pavlos Vougiouklis +4
While Wikipedia exists in 287 languages, its content is unevenly distributed among them. In this work, we investigate the generation of open domain Wikipedia summaries in underserv…
Zero-Shot Question Generation from Knowledge Graphs for Unseen Predicates and Entity Types
Hady Elsahar, Christophe Gravier, Frederique Laforest
We present a neural model for question generation from knowledge base triples in a "Zero-Shot" setup, that is generating questions for triples containing predicates, subject types…
Unsupervised Open Relation Extraction
Hady Elsahar, Elena Demidova, Simon Gottschalk +2
We explore methods to extract relations between named entities from free text in an unsupervised setting. In addition to standard feature extraction, we develop a novel method to r…
Neural Wikipedian: Generating Textual Summaries from Knowledge Base Triples
Pavlos Vougiouklis, Hady Elsahar, Lucie-Aimée Kaffee +4
Most people do not interact with Semantic Web data directly. Unless they have the expertise to understand the underlying technology, they need textual or visual interfaces to help…