4 citations · 4 across the 7 of their papers we have counts for
8 papers
Gradient Routing Localizes and Removes Unintended Behaviors in RL
Jake Ward, Shawn Hu, Aria Wong +4
We propose a mitigation to reward hacking in reinforcement learning. Our setup involves a misspecified reward function that gives rise to unintended behavior, and a monitor used to…
Mechanistically Eliciting Latent Behaviors in Language Models
Andrew Mack, Nina Panickssery, Alexander Matt Turner
We aim to discover diverse, generalizable perturbations of LLM internals that can surface hidden behavioral modes. Such perturbations could help reshape model behavior and systemat…
Is your algorithm unlearning or untraining?
Eleni Triantafillou, Ahmed Imtiaz Humayun, Monica Ribero +3
As models are getting larger and are trained on increasing amounts of data, there has been an explosion of interest into how we can ``delete'' specific data points or behaviours fr…
Recontextualization Mitigates Specification Gaming without Modifying the Specification
Ariana Azarbal, Victor Gillioz, Vladimir Ivanov +6
Developers often struggle to specify correct training labels and rewards. Perhaps they don't need to. We propose recontextualization, which reduces how often language models "game"…
Output Supervision Can Obfuscate the Chain of Thought
Jacob Drori, Luke Marks, Bryce Woodworth +2
OpenAI (2025) showed that training against a chain of thought (CoT) monitor can cause obfuscated CoTs, which contain bad behavior the monitor cannot detect. They proposed to keep C…
Distillation Robustifies Unlearning
Bruce W. Lee, Addie Foote, Alex Infanger +6
Current LLM unlearning methods are not robust. A few steps of finetuning can revert their effects. We begin by showing that this is true even for an idealized form of unlearning: t…
Gradient Routing: Masking Gradients to Localize Computation in Neural Networks
Alex Cloud, Jacob Goldman-Wetzler, Evžen Wybitul +2
Neural networks are trained primarily based on their inputs and outputs, without regard for their internal mechanisms. These neglected mechanisms determine properties that are crit…
Steering Llama 2 via Contrastive Activation Addition
Nina Panickssery, Nick Gabrieli, Julian Schulz +3
We introduce Contrastive Activation Addition (CAA), an innovative method for steering language models by modifying their activations during forward passes. CAA computes "steering v…