4 papers
Cluster-norm for Unsupervised Probing of Knowledge
Walter Laurito, Sharan Maiya, Grégoire Dhimoïla +3
The deployment of language models brings challenges in generating reliable information, especially when these models are fine-tuned using human preferences. To extract encoded know…
Mathematical Models of Computation in Superposition
Kaarel Hänni, Jake Mendel, Dmitry Vaintrob +1
Superposition -- when a neural network represents more ``features'' than it has dimensions -- seems to pose a serious challenge to mechanistically interpreting current AI systems.…
The Local Interaction Basis: Identifying Computationally-Relevant and Sparsely Interacting Features in Neural Networks
Lucius Bushnaq, Stefan Heimersheim, Nicholas Goldowsky-Dill +7
Mechanistic interpretability aims to understand the behavior of neural networks by reverse-engineering their internal computations. However, current methods struggle to find clear…
Using Degeneracy in the Loss Landscape for Mechanistic Interpretability
Lucius Bushnaq, Jake Mendel, Stefan Heimersheim +5
Mechanistic Interpretability aims to reverse engineer the algorithms implemented by neural networks by studying their weights and activations. An obstacle to reverse engineering ne…