Controlled LLM Training on Spectral Sphere

Tian Xie, Haoming Luo, Haoyu Tang +9

Scaling large models requires optimization strategies that ensure rapid convergence grounded in stability. Maximal Update Parametrization ($\hat{\mathbf{I}}¼$P) provides a theoretical…