Showing cs.LGShow all
2 papers · 1 filter
cs.LG2026
Demystifying Manifold Constraints in LLM Pre-training
Kang An, Jiaxiang Li, Donald Goldfarb +1
The recent success of matrix optimizers (e.g., Muon) suggests that specific normalization of momentum, such as orthogonalization and row-wise normalization, benefits both the stabi…
cs.LG2025
ASGO: Adaptive Structured Gradient Optimization
Kang An, Yuxing Liu, Rui Pan +4
Training deep neural networks is a structured optimization problem, because the parameters are naturally represented by matrices and tensors rather than by vectors. Under this stru…