Extracting Rule-based Descriptions of Attention Features in Transformers

Dan Friedman, Adithya Bhaskar, Alexander Wettig +1

Mechanistic interpretability strives to explain model behavior in terms of bottom-up primitives. The leading paradigm is to express hidden states as a sparse linear combination of…