Steering LLMs? Actually, Sparse Autoencoders can outperform simple baselines

Mikkel Godsk Jørgensen, Lars Kai Hansen

Sparse Autoencoders (SAEs) have been seen as a promising avenue for exploring the internals of Large Language Models (LLMs) and for steering model output generation. When AxBench -…