6 citations · 9 across the 23 of their papers we have counts for
9 papers · 1 filter
How AI settled the complexity of the oldest SGD algorithm
Michał Dereziński, Xiaoyu Dong
In 1937, Stefan Kaczmarz proposed a simple algorithm for solving systems of linear equations. This algorithm turned out to be the earliest known example of stochastic gradient desc…
Towards Universal Convergence of Backward Error in Linear System Solvers
Michał Dereziński, Yuji Nakatsukasa, Elizaveta Rebrova
The quest for an algorithm that solves an linear system in time complexity, or when solving up to relative error, is a long-stand…
Perfect Parallelization in Mini-Batch SGD with Classical Momentum Acceleration
Sachin Garg, Michał Dereziński
Accelerating stochastic gradient methods with classical momentum schemes, such as Polyak's heavy ball, has proven highly successful in training large-scale machine learning models,…
Numerical Instabilities in the Kaczmarz Method and Stabilization by Iterative Refinement
Michał Dereziński, Ethan N. Epperly, Deanna Needell +1
The randomized Kaczmarz method and its accelerated variants are a powerful class of iterative methods for solving large-scale linear systems, offering guaranteed convergence with l…
Accelerating Power Method with Fast Sketching for Stronger Low-Rank Approximation
Shabarish Chenakkod, Michał Dereziński
The power method is one of the most fundamental tools for extracting top principal components from data through low-rank matrix approximation. Yet, when the target rank is large, t…
Well-Conditioned Oblivious Perturbations in Linear Space
Shabarish Chenakkod, Michał Dereziński, Xiaoyu Dong +1
Perturbing a deterministic -dimensional matrix with small Gaussian noise is a cornerstone of smoothed analysis of algorithms [Spielman and Teng, JACM 2004], as it reduces the co…
Last-Iterate Convergence of Randomized Kaczmarz and SGD with Greedy Step Size
Michał Dereziński, Xiaoyu Dong
We study last-iterate convergence of SGD with greedy step size over smooth quadratics in the interpolation regime, a setting which captures the classical Randomized Kaczmarz algori…
Linear Systems and Eigenvalue Problems: Open Questions from a Simons Workshop
Noah Amsel, Yves Baumann, Paul Beckman +33
This document presents a series of open questions arising in matrix computations, i.e., the numerical solution of linear algebra problems. It is a result of working groups at the w…
The matrix-vector complexity of
Michał Dereziński, Ethan N. Epperly, Raphael A. Meyer
Matrix--vector algorithms, particularly Krylov subspace methods, are widely viewed as the most effective algorithms for solving large systems of linear equations. This paper establ…