4.3k citations · 6.1k across the 19 of their papers we have counts for
11 papers · 1 filter
Excess Description Length of Learning Generalizable Predictors
Elizabeth Donoway, Hailey Joren, Fabien Roger +1
Understanding whether fine-tuning elicits latent capabilities or teaches new ones is a fundamental question for language model evaluation and safety. We develop a formal informatio…
Forecasting Rare Language Model Behaviors
Erik Jones, Meg Tong, Jesse Mu +7
Standard language model evaluations can fail to capture risks that emerge only at deployment scale. For example, a model may produce safe responses during a small-scale beta test,…
Safe Deep RL in 3D Environments using Human Feedback
Matthew Rahtz, Vikrant Varma, Ramana Kumar +3
Agents should avoid unsafe behaviour during both training and deployment. This typically requires a simulator and a procedural specification of unsafe behaviour. Unfortunately, a s…
Evaluating Large Language Models Trained on Code
Mark Chen, Jerry Tworek, Heewoo Jun +55
We introduce Codex, a GPT language model fine-tuned on publicly available code from GitHub, and study its Python code-writing capabilities. A distinct production version of Codex p…
Active Reinforcement Learning: Observing Rewards at a Cost
David Krueger, Jan Leike, Owain Evans +1
Active reinforcement learning (ARL) is a variant on reinforcement learning where the agent does not observe the reward unless it chooses to pay a query cost c > 0. The central ques…
Hidden Incentives for Auto-Induced Distributional Shift
David Krueger, Tegan Maharaj, Jan Leike
Decisions made by machine learning systems have increasing influence on the world, yet it is common for machine learning algorithms to assume that no such influence exists. An exam…