68 citations · 241 across the 19 of their papers we have counts for
4 papers · 1 filter
On Reinforcement Learning and Distribution Matching for Fine-Tuning Language Models with no Catastrophic Forgetting
Tomasz Korbak, Hady Elsahar, Germán Kruszewski +1
The availability of large pre-trained models is changing the landscape of Machine Learning research and practice, moving from a training-from-scratch to a fine-tuning paradigm. Whi…
Sampling from Discrete Energy-Based Models with Quality/Efficiency Trade-offs
Bryan Eikema, Germán Kruszewski, Hady Elsahar +1
Energy-Based Models (EBMs) allow for extremely flexible specifications of probability distributions. However, they do not provide a mechanism for obtaining exact samples from these…
Controlling Conditional Language Models without Catastrophic Forgetting
Tomasz Korbak, Hady Elsahar, German Kruszewski +1
Machine learning is shifting towards general-purpose pretrained generative models, trained in a self-supervised manner on large amounts of data, which can then be applied to solve…
Energy-Based Models for Code Generation under Compilability Constraints
Tomasz Korbak, Hady Elsahar, Marc Dymetman +1
Neural language models can be successfully trained on source code, leading to applications such as code completion. However, their versatile autoregressive self-supervision objecti…