90 citations · 206 across the 39 of their papers we have counts for
22 papers · 1 filter
The Obfuscation Atlas: Mapping Where Honesty Emerges in RLVR with Deception Probes
Mohammad Taufeeque, Stefan Heimersheim, Adam Gleave +1
Training against white-box deception detectors has been proposed as a way to make AI systems honest. However, such training risks models learning to obfuscate their deception to ev…
Path Channels and Plan Extension Kernels: a Mechanistic Description of Planning in a Sokoban RNN
Mohammad Taufeeque, Aaron David Tucker, Adam Gleave +1
We partially reverse-engineer a convolutional recurrent neural network (RNN) trained with model-free reinforcement learning to play the box-pushing game Sokoban. We find that the R…
Preference Learning with Lie Detectors can Induce Honesty or Evasion
Chris Cundy, Adam Gleave
As AI systems become more capable, deceptive behaviors can undermine evaluation and mislead users at deployment. Recent work has shown that lie detectors can accurately classify de…
Scaling Trends in Language Model Robustness
Nikolaus Howe, Ian McKenzie, Oskar Hollinsworth +5
Increasing model size has unlocked a dazzling array of capabilities in modern language models. At the same time, even frontier models remain vulnerable to jailbreaks and prompt inj…
Planning in a recurrent neural network that plays Sokoban
Mohammad Taufeeque, Philip Quirke, Maximilian Li +4
Planning is essential for solving complex tasks, yet the internal mechanisms underlying planning in neural networks remain poorly understood. Building on prior work, we analyze a r…
Can Go AIs be adversarially robust?
Tom Tseng, Euan McLean, Kellin Pelrine +2
Prior work found that superhuman Go AIs can be defeated by simple adversarial strategies, especially "cyclic" attacks. In this paper, we study whether adding natural countermeasure…
STARC: A General Framework For Quantifying Differences Between Reward Functions
Joar Skalse, Lucy Farnik, Sumeet Ramesh Motwani +3
In order to solve a task using reinforcement learning, it is necessary to first formalise the goal of that task as a reward function. However, for many real-world tasks, it is very…
On The Fragility of Learned Reward Functions
Lev McKinney, Yawen Duan, David Krueger +1
Reward functions are notoriously difficult to specify, especially for tasks with complex goals. Reward learning approaches attempt to infer reward functions from human feedback and…
imitation: Clean Imitation Learning Implementations
Adam Gleave, Mohammad Taufeeque, Juan Rocamonde +7
imitation provides open-source implementations of imitation and reward learning algorithms in PyTorch. We include three inverse reinforcement learning (IRL) algorithms, three imita…
Adversarial Policies Beat Superhuman Go AIs
Tony T. Wang, Adam Gleave, Tom Tseng +8
We attack the state-of-the-art Go-playing AI system KataGo by training adversarial policies against it, achieving a >97% win rate against KataGo running at superhuman settings. Our…
Calculus on MDPs: Potential Shaping as a Gradient
Erik Jenner, Herke van Hoof, Adam Gleave
In reinforcement learning, different reward functions can be equivalent in terms of the optimal policies they induce. A particularly well-known and important example is potential s…
Reducing Exploitability with Population Based Training
Pavel Czempin, Adam Gleave
Self-play reinforcement learning has achieved state-of-the-art, and often superhuman, performance in a variety of zero-sum games. Yet prior work has found that policies that are hi…
Preprocessing Reward Functions for Interpretability
Erik Jenner, Adam Gleave
In many real-world applications, the reward function is too complex to be manually specified. In such cases, reward functions must instead be learned from human feedback. Since the…
A Primer on Maximum Causal Entropy Inverse Reinforcement Learning
Adam Gleave, Sam Toyer
Inverse Reinforcement Learning (IRL) algorithms infer a reward function that explains demonstrations provided by an expert acting in the environment. Maximum Causal Entropy (MCE) I…
Invariance in Policy Optimisation and Partial Identifiability in Reward Learning
Joar Skalse, Matthew Farrugia-Roberts, Stuart Russell +2
It is often very challenging to manually design reward functions for complex, real-world tasks. To solve this, one can instead use reward learning to infer a reward function from d…
Understanding Learned Reward Functions
Eric J. Michaud, Adam Gleave, Stuart Russell
In many real-world tasks, it is not possible to procedurally specify an RL agent's reward function. In such cases, a reward function must instead be learned from interacting with a…
DERAIL: Diagnostic Environments for Reward And Imitation Learning
Pedro Freire, Adam Gleave, Sam Toyer +1
The objective of many real-world tasks is complex and difficult to procedurally specify. This makes it necessary to use reward or imitation learning algorithms to infer a reward or…
Quantifying Differences in Reward Functions
Adam Gleave, Michael Dennis, Shane Legg +2
For many tasks, the reward function is inaccessible to introspection or too complex to be specified procedurally, and must instead be learned from user data. Prior work has evaluat…
Adversarial Policies: Attacking Deep Reinforcement Learning
Adam Gleave, Michael Dennis, Cody Wild +3
Deep reinforcement learning (RL) policies are known to be vulnerable to adversarial perturbations to their observations, similar to adversarial examples for classifiers. However, a…
Inverse reinforcement learning for video games
Aaron Tucker, Adam Gleave, Stuart Russell
Deep reinforcement learning achieves superhuman performance in a range of video game environments, but requires that a designer manually specify a reward function. It is often easi…
Active Inverse Reward Design
Sören Mindermann, Rohin Shah, Adam Gleave +1
Designers of AI agents often iterate on the reward function in a trial-and-error process until they get the desired behavior, but this only guarantees good behavior in the training…
Multi-task Maximum Entropy Inverse Reinforcement Learning
Adam Gleave, Oliver Habryka
Multi-task Inverse Reinforcement Learning (IRL) is the problem of inferring multiple reward functions from expert demonstrations. Prior work, built on Bayesian IRL, is unable to sc…