22 citations · 51 across the 7 of their papers we have counts for
11 papers
What Language Model to Train if You Have One Million GPU Hours?
Teven Le Scao, Thomas Wang, Daniel Hesslow +16
The crystallization of modeling methods around the Transformer architecture has been a boon for practitioners. Simple, well-motivated architectural variations can transfer across t…
Documenting Geographically and Contextually Diverse Data Sources: The BigScience Catalogue of Language Data and Resources
Angelina McMillan-Major, Zaid Alyafeai, Stella Biderman +15
In recent years, large-scale data collection efforts have prioritized the amount of data collected in order to improve the modeling capabilities of large language models. This prio…
Unsupervised and Distributional Detection of Machine-Generated Text
Matthias Gallé, Jos Rozen, Germán Kruszewski +1
The power of natural language generation models has provoked a flurry of interest in automatic methods to detect if a piece of text is human or machine-authored. The problem so far…
Energy-Based Models for Code Generation under Compilability Constraints
Tomasz Korbak, Hady Elsahar, Marc Dymetman +1
Neural language models can be successfully trained on source code, leading to applications such as code completion. However, their versatile autoregressive self-supervision objecti…
References in Wikipedia: The Editors' Perspective
Lucie-Aimée Kaffee, Hady Elsahar
References are an essential part of Wikipedia. Each statement in Wikipedia should be referenced. In this paper, we explore the creation and collection of references for new Wikiped…
A Distributional Approach to Controlled Text Generation
Muhammad Khalifa, Hady Elsahar, Marc Dymetman
We propose a Distributional Approach for addressing Controlled Text Generation from pre-trained Language Models (LMs). This approach permits to specify, in a single formal framewor…