Showing math.OCShow all
2 papers · 1 filter
math.OC2026
Convergence Rate Analysis of SOAP with Arbitrary Orthogonal Projection Matrices
Huan Li, Zhouchen Lin
In this short note, we establish, for the first time, the convergence rate of SOAP, an efficient and popular matrix-based optimizer for training deep neural networks. Our analysis…
math.OC2018
Decentralized Accelerated Gradient Methods With Increasing Penalty Parameters
Huan Li, Cong Fang, Wotao Yin +1
In this paper, we study the communication and (sub)gradient computation costs in distributed optimization and give a sharp complexity analysis for the proposed distributed accelera…