51 citations · 51 across the 3 of their papers we have counts for
3 papers
cs.AI2026
Polysemantic Experts, Monosemantic Paths: Routing as Control in MoEs
Charles Ye, Bo Yuan, Lee Sharkey
An LLM's residual stream is both state and instruction: it encodes the current context and determines the next transformation. We introduce a parameter-free decomposition for Mixtu…
cs.LG2025
Sparse Autoencoders Do Not Find Canonical Units of Analysis
Patrick Leask, Bart Bussmann, Michael Pearce +5
A common goal of mechanistic interpretability is to decompose the activations of neural networks into features: interpretable properties of the input computed by the model. Sparse…
cs.LG2023★ 51 cited
Sparse Autoencoders Find Highly Interpretable Features in Language Models
Hoagy Cunningham, Aidan Ewart, Logan Riggs +2
One of the roadblocks to a better understanding of neural networks' internals is \textit{polysemanticity}, where neurons appear to activate in multiple, semantically distinct conte…