Evolution of SAE Features Across Layers in LLMs

Daniel Balcells, Benjamin Lerner, Michael Oesterle +2

Sparse Autoencoders for transformer-based language models are typically defined independently per layer. In this work we analyze statistical relationships between features in adjac…