6.9k citations · 24.1k across the 46 of their papers we have counts for
3 papers · 1 filter
Fine-Tuning Language Models from Human Preferences
Daniel M. Ziegler, Nisan Stiennon, Jeffrey Wu +5
Reward learning enables the application of reinforcement learning (RL) to tasks where reward is defined by human judgment, building a model of reward by asking humans questions. Mo…
Release Strategies and the Social Impacts of Language Models
Irene Solaiman, Miles Brundage, Jack Clark +12
Large language models have a range of beneficial uses: they can assist in prose, poetry, and programming; analyze dataset biases; and more. However, their flexibility and generativ…
Generating Long Sequences with Sparse Transformers
Rewon Child, Scott Gray, Alec Radford +1
Transformers are powerful sequence models, but require time and memory that grows quadratically with the sequence length. In this paper we introduce sparse factorizations of the at…