3 papers
cs.LG2026
Why Loss Re-weighting Works If You Stop Early: Training Dynamics of Unconstrained Features
Yize Zhao, Christos Thrampoulidis
The application of loss reweighting in modern deep learning presents a nuanced picture. While it fails to alter the terminal learning phase in overparameterized deep neural network…
cs.LG2025
How Muon's Spectral Design Benefits Generalization: A Study on Imbalanced Data
Bhavya Vasudeva, Puneesh Deora, Yize Zhao +2
The growing adoption of spectrum-aware matrix-valued optimizers such as Muon and Shampoo in deep learning motivates a systematic study of their generalization properties and, in pa…
cs.CL2025
Geometry of Semantics in Next-Token Prediction: How Optimization Implicitly Organizes Linguistic Representations
Yize Zhao, Christos Thrampoulidis
We investigate how next-token prediction (NTP) optimization leads language models to extract and organize semantic structure from text. Our analysis, based on a tractable mathemati…