23 citations · 28 across the 4 of their papers we have counts for
4 papers
A technical note on bilinear layers for interpretability
Lee Sharkey
The ability of neural networks to represent more features than neurons makes interpreting them challenging. This phenomenon, known as superposition, has spurred efforts to find arc…
Circumventing interpretability: How to defeat mind-readers
Lee Sharkey
The increasing capabilities of artificial intelligence (AI) systems make it ever more important that we interpret their internals to ensure that their intentions are aligned with h…
Interpreting Neural Networks through the Polytope Lens
Sid Black, Lee Sharkey, Leo Grinsztajn +8
Mechanistic interpretability aims to explain what a neural network has learned at a nuts-and-bolts level. What are the fundamental primitives of neural network representations? Pre…
Goal Misgeneralization in Deep Reinforcement Learning
Lauro Langosco, Jack Koch, Lee Sharkey +3
We study goal misgeneralization, a type of out-of-distribution generalization failure in reinforcement learning (RL). Goal misgeneralization failures occur when an RL agent retains…