Inference and learning in sparse autoencoders as natural gradient flow

Hadi Vafaii, Tejas Rao, David Chanin +6

Sparse autoencoders are widely used to uncover interpretable features in neural networks, yet reliable recovery remains difficult when features overlap or activate infrequently. Th…