1 citations · 2 across the 6 of their papers we have counts for
3 papers · 1 filter
Effective Quantization of Muon Optimizer States
Aman Gupta, Rafael Celente, Abhishek Shivanna +7
The Muon optimizer, based on matrix orthogonalization, has recently shown faster convergence and better computational efficiency over AdamW in LLM pre-training. However, the memory…
LiRank: Industrial Large Scale Ranking Models at LinkedIn
Fedor Borisyuk, Mingzhou Zhou, Qingquan Song +31
We present LiRank, a large-scale ranking framework at LinkedIn that brings to production state-of-the-art modeling architectures and optimization methods. We unveil several modelin…
A Precise Characterization of SGD Stability Using Loss Surface Geometry
Gregory Dexter, Borja Ocejo, Sathiya Keerthi +3
Stochastic Gradient Descent (SGD) stands as a cornerstone optimization algorithm with proven real-world empirical successes but relatively limited theoretical understanding. Recent…