100 citations · 462 across the 26 of their papers we have counts for
3 papers · 1 filter
Drop-Muon: Update Less, Converge Faster
Kaja Gruntkowska, Yassine Maziane, Zheng Qu +1
Conventional wisdom in deep learning optimization dictates updating all layers at every step-a principle followed by all recent state-of-the-art optimizers such as Muon. In this wo…
SAGA with Arbitrary Sampling
Xu Qian, Zheng Qu, Peter Richtárik
We study the problem of minimizing the average of a very large number of smooth functions, which is of key importance in training supervised learning models. One of the most celebr…
SDNA: Stochastic Dual Newton Ascent for Empirical Risk Minimization
Zheng Qu, Peter Richtárik, Martin Takáč +1
We propose a new algorithm for minimizing regularized empirical loss: Stochastic Dual Newton Ascent (SDNA). Our method is dual in nature: in each iteration we update a random subse…