Showing cs.LGShow all
2 papers · 1 filter
cs.LG2026
Dead-Direction Signatures: A Cheap Spectral Reading of Singular Complexity
Tejas Pradeep Shirodkar, P. J. Narayanan
Singular learning theory characterises the complexity of a deep network through the geometry of its loss singularities. The local learning coefficient (LLC), the standard estimator…
cs.LG2026
Algebraic Dead Directions in LayerNorm Transformers: A Forward-Pass-Only Diagnostic at LLM Scale
Tejas Pradeep Shirodkar, P. J. Narayanan
Pretrained transformers sit near singular minima of the loss, where the Fisher information metric degenerates along dead directions: directions in parameter space along which the d…