Non-Euclidean SGD for Structured Optimization: Unified Analysis and Improved Rates

Dmitry Kovalev, Ekaterina Borodich

Recently, several instances of non-Euclidean SGD, including SignSGD, Lion, and Muon, have attracted significant interest from the optimization community due to their practical succ…