Structure Before Collapse: Transient semantic geometry in next-token prediction

Yize Zhao, Isabel Papadimitriou, Christos Thrampoulidis

Neural Collapse predicts that balanced one-hot classification pushes model representations to be equally far from each other; a symmetric configuration that depends only on the out…