Showing cs.LGShow all
3 papers · 1 filter
cs.LG2026
When to use what Schatten- norm in deep learning?
Thomas Pethick
Schatten- based optimizers such as Muon have shown promising empirical performance, but there remains seemingly conflicting observations regarding whether they are benefici…
cs.LG2025
Training Neural Networks at Any Scale
Thomas Pethick, Kimon Antonakopoulos, Antonio Silveti-Falls +2
This article reviews modern optimization methods for training neural networks with an emphasis on efficiency and scale. We present state-of-the-art optimization algorithms under a…
cs.LG2025
Generalized Gradient Norm Clipping & Non-Euclidean -Smoothness
Thomas Pethick, Wanyun Xie, Mete Erdogan +3
This work introduces a hybrid non-Euclidean optimization method which generalizes gradient norm clipping by combining steepest descent and conditional gradient approaches. The meth…