106 citations · 160 across the 17 of their papers we have counts for
14 papers · 1 filter
BLOOM: A 176B-Parameter Open-Access Multilingual Language Model
BigScience Workshop, :, Teven Le Scao +391
Large language models (LLMs) have been shown to be able to perform new tasks based on a few demonstrations or natural language instructions. While these capabilities have led to wi…
Between words and characters: A Brief History of Open-Vocabulary Modeling and Tokenization in NLP
Sabrina J. Mielke, Zaid Alyafeai, Elizabeth Salesky +8
What are the units of text that we want to model? From bytes to multi-word expressions, text can be analyzed and generated at many granularities. Until recently, most natural langu…
Unsupervised and Distributional Detection of Machine-Generated Text
Matthias Gallé, Jos Rozen, Germán Kruszewski +1
The power of natural language generation models has provoked a flurry of interest in automatic methods to detect if a piece of text is human or machine-authored. The problem so far…
Speeding Up Entmax
Maxat Tezekbayev, Vassilina Nikoulina, Matthias Gallé +1
Softmax is the de facto standard in modern neural networks for language processing when it comes to normalizing logits. However, by producing a dense probability distribution each…
Multilingual Unsupervised Neural Machine Translation with Denoising Adapters
Ahmet Üstün, Alexandre Bérard, Laurent Besacier +1
We consider the problem of multilingual unsupervised machine translation, translating to and from languages that only have monolingual data by using auxiliary parallel language pai…
On the Evaluation of Machine Translation for Terminology Consistency
Md Mahfuz ibn Alam, Antonios Anastasopoulos, Laurent Besacier +4
As neural machine translation (NMT) systems become an important part of professional translator pipelines, a growing body of work focuses on combining NMT with terminologies. In ma…
Breaking Writer's Block: Low-cost Fine-tuning of Natural Language Generation Models
Alexandre Duval, Thomas Lamson, Gael de Leseleuc de Kerouara +1
It is standard procedure these days to solve Information Extraction task by fine-tuning large pre-trained language models. This is not the case for generation task, which relies on…
The Rediscovery Hypothesis: Language Models Need to Meet Linguistics
Vassilina Nikoulina, Maxat Tezekbayev, Nuradil Kozhakhmet +3
There is an ongoing debate in the NLP community whether modern language models contain linguistic knowledge, recovered through so-called probes. In this paper, we study whether lin…
A Multilingual Neural Machine Translation Model for Biomedical Data
Alexandre Bérard, Zae Myung Kim, Vassilina Nikoulina +2
We release a multilingual neural machine translation model, which can be used to translate text in the biomedical domain. The model can translate from 5 languages (French, German,…
Self-Supervised and Controlled Multi-Document Opinion Summarization
Hady Elsahar, Maximin Coavoux, Matthias Gallé +1
We address the problem of unsupervised abstractive summarization of collections of user generated reviews with self-supervision and control. We propose a self-supervised setup that…
Character-based NMT with Transformer
Rohit Gupta, Laurent Besacier, Marc Dymetman +1
Character-based translation has several appealing advantages, but its performance is in general worse than a carefully tuned BPE baseline. In this paper we study the impact of char…
The Generalized Smallest Grammar Problem
Payam Siyari, Matthias Gallé
The Smallest Grammar Problem -- the problem of finding the smallest context-free grammar that generates exactly one given sequence -- has never been successfully applied to grammat…
Discriminating between similar languages in Twitter using label propagation
Will Radford, Matthias Galle
Identifying the language of social media messages is an important first step in linguistic processing. Existing models for Twitter focus on content analysis, which is successful fo…
Joint Event Detection and Entity Resolution: a Virtuous Cycle
Matthias Galle, Jean-Michel Renders, Guillaume Jacquet
Clustering web documents has numerous applications, such as aggregating news articles into meaningful events, detecting trends and hot topics on the Web, preserving diversity in se…