106 citations · 160 across the 17 of their papers we have counts for
20 papers
Back to Basics: Revisiting REINFORCE Style Optimization for Learning from Human Feedback in LLMs
Arash Ahmadian, Chris Cremer, Matthias Gallé +5
AI alignment in the shape of Reinforcement Learning from Human Feedback (RLHF) is increasingly treated as a crucial ingredient for high performance large language models. Proximal…
BigScience: A Case Study in the Social Construction of a Multilingual Large Language Model
Christopher Akiki, Giada Pistilli, Margot Mieskes +4
The BigScience Workshop was a value-driven initiative that spanned one and half years of interdisciplinary research and culminated in the creation of ROOTS, a 1.6TB multilingual da…
BLOOM: A 176B-Parameter Open-Access Multilingual Language Model
BigScience Workshop, :, Teven Le Scao +391
Large language models (LLMs) have been shown to be able to perform new tasks based on a few demonstrations or natural language instructions. While these capabilities have led to wi…
Between words and characters: A Brief History of Open-Vocabulary Modeling and Tokenization in NLP
Sabrina J. Mielke, Zaid Alyafeai, Elizabeth Salesky +8
What are the units of text that we want to model? From bytes to multi-word expressions, text can be analyzed and generated at many granularities. Until recently, most natural langu…
Unsupervised and Distributional Detection of Machine-Generated Text
Matthias Gallé, Jos Rozen, Germán Kruszewski +1
The power of natural language generation models has provoked a flurry of interest in automatic methods to detect if a piece of text is human or machine-authored. The problem so far…
Speeding Up Entmax
Maxat Tezekbayev, Vassilina Nikoulina, Matthias Gallé +1
Softmax is the de facto standard in modern neural networks for language processing when it comes to normalizing logits. However, by producing a dense probability distribution each…
Multilingual Unsupervised Neural Machine Translation with Denoising Adapters
Ahmet Üstün, Alexandre Bérard, Laurent Besacier +1
We consider the problem of multilingual unsupervised machine translation, translating to and from languages that only have monolingual data by using auxiliary parallel language pai…
On the Evaluation of Machine Translation for Terminology Consistency
Md Mahfuz ibn Alam, Antonios Anastasopoulos, Laurent Besacier +4
As neural machine translation (NMT) systems become an important part of professional translator pipelines, a growing body of work focuses on combining NMT with terminologies. In ma…
Breaking Writer's Block: Low-cost Fine-tuning of Natural Language Generation Models
Alexandre Duval, Thomas Lamson, Gael de Leseleuc de Kerouara +1
It is standard procedure these days to solve Information Extraction task by fine-tuning large pre-trained language models. This is not the case for generation task, which relies on…
The Rediscovery Hypothesis: Language Models Need to Meet Linguistics
Vassilina Nikoulina, Maxat Tezekbayev, Nuradil Kozhakhmet +3
There is an ongoing debate in the NLP community whether modern language models contain linguistic knowledge, recovered through so-called probes. In this paper, we study whether lin…
A Multilingual Neural Machine Translation Model for Biomedical Data
Alexandre Bérard, Zae Myung Kim, Vassilina Nikoulina +2
We release a multilingual neural machine translation model, which can be used to translate text in the biomedical domain. The model can translate from 5 languages (French, German,…
Self-Supervised and Controlled Multi-Document Opinion Summarization
Hady Elsahar, Maximin Coavoux, Matthias Gallé +1
We address the problem of unsupervised abstractive summarization of collections of user generated reviews with self-supervision and control. We propose a self-supervised setup that…
Character-based NMT with Transformer
Rohit Gupta, Laurent Besacier, Marc Dymetman +1
Character-based translation has several appealing advantages, but its performance is in general worse than a carefully tuned BPE baseline. In this paper we study the impact of char…
A Maximum Matching Algorithm for Basis Selection in Spectral Learning
Ariadna Quattoni, Xavier Carreras, Matthias Gallé
We present a solution to scale spectral algorithms for learning sequence functions. We are interested in the case where these functions are sparse (that is, for most sequences they…
What Can I Do Now? Guiding Users in a World of Automated Decisions
Matthias Gallé
More and more processes governing our lives use in some part an automatic decision step, where -- based on a feature vector derived from an applicant -- an algorithm has the decisi…
The Generalized Smallest Grammar Problem
Payam Siyari, Matthias Gallé
The Smallest Grammar Problem -- the problem of finding the smallest context-free grammar that generates exactly one given sequence -- has never been successfully applied to grammat…
Discriminating between similar languages in Twitter using label propagation
Will Radford, Matthias Galle
Identifying the language of social media messages is an important first step in linguistic processing. Existing models for Twitter focus on content analysis, which is successful fo…
Multi-view pattern matching
Matthias Galle
We introduce the \textit{multi-view pattern matching} problem, where a text can have multiple views. Each view is a string of the same size and drawn from disjoint alphabets. The p…
Joint Event Detection and Entity Resolution: a Virtuous Cycle
Matthias Galle, Jean-Michel Renders, Guillaume Jacquet
Clustering web documents has numerous applications, such as aggregating news articles into meaningful events, detecting trends and hot topics on the Web, preserving diversity in se…
"Roles for the boys?" Mining cast lists for gender and role distributions over time
William Radford, Matthias Gallé
Film and television play an important role in popular culture, however studies that require watching and annotating video are time-consuming and expensive to run at scale. We explo…