11 papers
Multi-Dimensional Featurizers Yield Fine-Grained Steering and Data Shaping in Robot Foundation Models
Aiden Swann, Mozes Jacobs, Mathilde Papillon +12
Can better feature discovery improve control over robot behavior through internal interventions and training data? Recent work in vision and language models reveals multidimensiona…
Inference and learning in sparse autoencoders as natural gradient flow
Hadi Vafaii, Tejas Rao, David Chanin +6
Sparse autoencoders are widely used to uncover interpretable features in neural networks, yet reliable recovery remains difficult when features overlap or activate infrequently. Th…
Are Sparse Autoencoder Benchmarks Reliable?
David Chanin
Sparse autoencoders (SAEs) are a core interpretability tool for large language models, and progress on SAE architectures depends on benchmarks that reliably distinguish better SAEs…
SynthSAEBench: Evaluating Sparse Autoencoders on Scalable Realistic Synthetic Data
David Chanin, Adrià Garriga-Alonso
Improving Sparse Autoencoders (SAEs) requires benchmarks that can precisely validate architectural innovations. Current LLM-based SAE benchmarks are too noisy to differentiate arch…
Biases in the Blind Spot: Detecting What LLMs Fail to Mention
Iván Arcuschin, David Chanin, Adrià Garriga-Alonso +1
Large Language Models (LLMs) often provide chain-of-thought (CoT) reasoning traces that appear plausible, but may hide internal biases. We call these unverbalized biases. Monitorin…
Sparse but Wrong: Incorrect L0 Leads to Incorrect Features in Sparse Autoencoders
David Chanin, Adrià Garriga-Alonso
Sparse Autoencoders (SAEs) extract features from LLM internal activations, meant to correspond to interpretable concepts. A core SAE training hyperparameter is L0: how many SAE fea…
Feature Hedging: Correlated Features Break Narrow Sparse Autoencoders
David Chanin, Tomáš Dulka, Adrià Garriga-Alonso
It is assumed that sparse autoencoders (SAEs) decompose polysemantic activations into interpretable linear directions, as long as the activations are composed of sparse linear comb…
SAEBench: A Comprehensive Benchmark for Sparse Autoencoders in Language Model Interpretability
Adam Karvonen, Can Rager, Johnny Lin +12
Sparse autoencoders (SAEs) are a popular technique for interpreting language model activations, and there is extensive recent work on improving SAE effectiveness. However, most pri…
A is for Absorption: Studying Feature Splitting and Absorption in Sparse Autoencoders
David Chanin, James Wilken-Smith, Tomáš Dulka +3
Sparse Autoencoders (SAEs) aim to decompose the activation space of large language models (LLMs) into human-interpretable latent directions or features. As we increase the number o…
Analyzing the Generalization and Reliability of Steering Vectors
Daniel Tan, David Chanin, Aengus Lynch +4
Steering vectors (SVs) have been proposed as an effective approach to adjust language model behaviour at inference time by intervening on intermediate model activations. They have…
Identifying Linear Relational Concepts in Large Language Models
David Chanin, Anthony Hunter, Oana-Maria Camburu
Transformer language models (LMs) have been shown to represent concepts as directions in the latent space of hidden activations. However, for any human-interpretable concept, how c…