95 citations · 201 across the 13 of their papers we have counts for
13 papers
Open Problems and Fundamental Limitations of Reinforcement Learning from Human Feedback
Stephen Casper, Xander Davies, Claudia Shi +29
Reinforcement learning from human feedback (RLHF) is a technique for training AI systems to align with human goals. RLHF has emerged as the central method used to finetune state-of…
Training Language Models with Language Feedback at Scale
Jérémy Scheurer, Jon Ander Campos, Tomasz Korbak +4
Pretrained language models often generate outputs that are not in line with human preferences, such as harmful text or factually incorrect summaries. Recent work approaches the abo…
Pretraining Language Models with Human Preferences
Tomasz Korbak, Kejian Shi, Angelica Chen +5
Language models (LMs) are pretrained to imitate internet text, including content that would violate human preferences if generated by an LM: falsehoods, offensive comments, persona…
Aligning Language Models with Preferences through f-divergence Minimization
Dongyoung Go, Tomasz Korbak, Germán Kruszewski +3
Aligning language models with preferences can be posed as approximating a target distribution representing some desired behavior. Existing approaches differ both in the functional…
On Reinforcement Learning and Distribution Matching for Fine-Tuning Language Models with no Catastrophic Forgetting
Tomasz Korbak, Hady Elsahar, Germán Kruszewski +1
The availability of large pre-trained models is changing the landscape of Machine Learning research and practice, moving from a training-from-scratch to a fine-tuning paradigm. Whi…
RL with KL penalties is better viewed as Bayesian inference
Tomasz Korbak, Ethan Perez, Christopher L Buckley
Reinforcement learning (RL) is frequently employed in fine-tuning large language models (LMs), such as GPT-3, to penalize them for undesirable features of generated sequences, such…