Sparsity as a Key: Unlocking New Insights from Latent Structures for Out-of-Distribution Detection

Ahyoung Oh, Wonseok Shin, Songkuk Kim

Sparse Autoencoders (SAEs) have demonstrated significant success in interpreting Large Language Models (LLMs) by decomposing dense representations into sparse, semantic components.…