101 citations · 488 across the 33 of their papers we have counts for
4 papers · 1 filter
Regularized Newton Method with Global Convergence
Konstantin Mishchenko
We present a Newton-type method that converges fast from any initialization and for arbitrary convex objectives with Lipschitz Hessians. We achieve this by merging the ideas of cub…
On Seven Fundamental Optimization Challenges in Machine Learning
Konstantin Mishchenko
Many recent successes of machine learning went hand in hand with advances in optimization. The exchange of ideas between these fields has worked both ways, with machine learning bu…
Proximal and Federated Random Reshuffling
Konstantin Mishchenko, Ahmed Khaled, Peter Richtárik
Random Reshuffling (RR), also known as Stochastic Gradient Descent (SGD) without replacement, is a popular and theoretically grounded method for finite-sum minimization. We propose…
IntSGD: Adaptive Floatless Compression of Stochastic Gradients
Konstantin Mishchenko, Bokun Wang, Dmitry Kovalev +1
We propose a family of adaptive integer compression operators for distributed Stochastic Gradient Descent (SGD) that do not communicate a single float. This is achieved by multiply…