95 citations · 201 across the 13 of their papers we have counts for
Showing 2022Show all
2 papers · 1 filter
cs.LG2022★ 6 cited
On Reinforcement Learning and Distribution Matching for Fine-Tuning Language Models with no Catastrophic Forgetting
Tomasz Korbak, Hady Elsahar, Germán Kruszewski +1
The availability of large pre-trained models is changing the landscape of Machine Learning research and practice, moving from a training-from-scratch to a fine-tuning paradigm. Whi…
cs.LG2022
RL with KL penalties is better viewed as Bayesian inference
Tomasz Korbak, Ethan Perez, Christopher L Buckley
Reinforcement learning (RL) is frequently employed in fine-tuning large language models (LMs), such as GPT-3, to penalize them for undesirable features of generated sequences, such…