9 citations · 19 across the 7 of their papers we have counts for
3 papers · 1 filter
In-Context Algebra
Eric Todd, Jannik Brinkmann, Rohit Gandikota +1
We investigate the mechanisms that arise when transformers are trained to solve arithmetic on sequences where tokens are variables whose meaning is determined only through their in…
The Dual-Route Model of Induction
Sheridan Feucht, Eric Todd, Byron Wallace +1
Prior work on in-context copying has shown the existence of induction heads, which attend to and promote individual tokens during copying. In this work we discover a new type of in…
Open Problems in Mechanistic Interpretability
Lee Sharkey, Bilal Chughtai, Joshua Batson +26
Mechanistic interpretability aims to understand the computational mechanisms underlying neural networks' capabilities in order to accomplish concrete scientific and engineering goa…