185 citations · 227 across the 9 of their papers we have counts for
15 papers
On Convergence of Incremental Gradient for Non-Convex Smooth Functions
Anastasia Koloskova, Nikita Doikov, Sebastian U. Stich +1
In machine learning and neural network optimization, algorithms like incremental gradient, and shuffle SGD are popular due to minimizing the number of cache misses and good practic…
Revisiting Gradient Clipping: Stochastic bias and tight convergence guarantees
Anastasia Koloskova, Hadrien Hendrikx, Sebastian U. Stich
Gradient clipping is a popular modification to standard (stochastic) gradient descent, at every iteration limiting the gradient norm to a certain value . It is widely used fo…
Gradient Descent with Linearly Correlated Noise: Theory and Applications to Differential Privacy
Anastasia Koloskova, Ryan McKenna, Zachary Charles +2
We study gradient descent under linearly correlated noise. Our work is motivated by recent practical methods for optimization with differential privacy (DP), such as DP-FTRL, which…
Decentralized Gradient Tracking with Local Steps
Yue Liu, Tao Lin, Anastasia Koloskova +1
Gradient tracking (GT) is an algorithm designed for solving decentralized optimization problems over a network (such as training a machine learning model). A key feature of GT is a…
Sharper Convergence Guarantees for Asynchronous SGD for Distributed and Federated Learning
Anastasia Koloskova, Sebastian U. Stich, Martin Jaggi
We study the asynchronous stochastic gradient descent algorithm for distributed training over workers which have varying computation and communication frequency over time. In t…
Data-heterogeneity-aware Mixing for Decentralized Learning
Yatin Dandi, Anastasia Koloskova, Martin Jaggi +1
Decentralized learning provides an effective framework to train machine learning models with data distributed over arbitrary communication graphs. However, most existing approaches…