3 papers
cs.LG2026
Demystifying Manifold Constraints in LLM Pre-training
Kang An, Jiaxiang Li, Donald Goldfarb +1
The recent success of matrix optimizers (e.g., Muon) suggests that specific normalization of momentum, such as orthogonalization and row-wise normalization, benefits both the stabi…
math.OC2025
Non-Convex Self-Concordant Functions: Practical Algorithms and Complexity Analysis
Donald Goldfarb, Lexiao Lai, Tianyi Lin +1
We extend the standard notion of self-concordance to non-convex optimization and develop a family of second-order algorithms with global convergence guarantees. In particular, two…
cs.LG2025
ASGO: Adaptive Structured Gradient Optimization
Kang An, Yuxing Liu, Rui Pan +4
Training deep neural networks is a structured optimization problem, because the parameters are naturally represented by matrices and tensors rather than by vectors. Under this stru…