Identifying Sparsely Active Circuits Through Local Loss Landscape Decomposition

Brianna Chrisman, Lucius Bushnaq, Lee Sharkey

Much of mechanistic interpretability has focused on understanding the activation spaces of large neural networks. However, activation space-based approaches reveal little about the…