2 papers
cs.LG2026
Concepts Whisper: Spectral Anti-Concentration and the Dual Geometry of Transformer Representations
Pratyush Acharya, Nuraj Rimal, Habish Dhakal
We find that transformer concept representations systematically anti-concentrate in the spectral tail of the unembedding covariance, encoding word-level concepts in low-variance di…
cs.LG2026
Grokking as a Variance-Limited Phase Transition: Spectral Gating and the Epsilon-Stability Threshold
Pratyush Acharya, Habish Dhakal
Standard optimization theories struggle to explain grokking, where generalization occurs long after training convergence. While geometric studies attribute this to slow drift, they…