5 papers
Gradient Routing Localizes and Removes Unintended Behaviors in RL
Jake Ward, Shawn Hu, Aria Wong +4
We propose a mitigation to reward hacking in reinforcement learning. Our setup involves a misspecified reward function that gives rise to unintended behavior, and a monitor used to…
Matryoshka attribution: Learning to attribute language model outputs to representations and weights
Aryaman Arora, Kirill Acharya, Nathan Hu +4
Attributing language model outputs to their internal computations is an open problem in interpretability. Existing methods, which use causal interventions, gradients, or learnable…
Verbalizing Subliminal Learning Effects Using Text Optimization
Nathan Hu, Sanmi Koyejo, Christopher Potts
Subliminal learning is a phenomenon in which a distillation dataset transmits traits from the teacher model that are not legibly encoded in the dataset itself. This introduces a ne…
Transcoder Adapters for Reasoning-Model Diffing
Nathan Hu, Jake Ward, Thomas Icard +1
While reasoning models are increasingly ubiquitous, the effects of reasoning training on a model's internal mechanisms remain poorly understood. In this work, we introduce transcod…
Confucius Code Agent: Scalable Agent Scaffolding for Real-World Codebases
Sherman Wong, Zhenting Qi, Zhaodong Wang +8
Real-world software engineering tasks require coding agents that can operate on massive repositories, sustain long-horizon sessions, and reliably coordinate complex toolchains at t…