6 papers · 1 filter
Quantile Randomized Kaczmarz for Streaming Linear Systems with Massart Noise
Emeric Battaglia, Jian-Feng Cai, Junren Chen +3
Quantile randomized Kaczmarz (QRK) has proven to be an efficient solver for corrupted linear systems and has received much attention. It was recently shown by Cai et al. (SIAM J. M…
RPLSS: A randomized projected linear systems solver
Meng-Long Xiao, Tao Li, Deanna Needell
The projected linear system solver (PLSS), by incrementally appending columns to a random or deterministic sketching matrix, provides an attractive finite termination property for…
Attention Mechanisms Through the Lens of Numerical Methods: Approximation Methods and Alternative Formulations
Michel Fabrice Serret, Alice Cortinovis, Yijun Dong +10
The attention mechanism is the computational core of modern Transformer architectures, but its quadratic complexity in the input sequence length is the bottleneck for large-scale i…
Stochastic Gradient Descent for Incomplete Tensor Linear Systems
Anna Ma, Deanna Needell, Alexander Xue
Solving large tensor linear systems poses significant challenges due to the high volume of data stored, and it only becomes more challenging when some of the data is missing. Recen…
A federated Kaczmarz algorithm
Halyun Jeong, Deanna Needell, Chi-Hao Wu
In this paper, we propose a federated algorithm for solving large linear systems that is inspired by the classic randomized Kaczmarz algorithm. We provide convergence guarantees of…
Randomized Kaczmarz Methods with Beyond-Krylov Convergence
Michał Dereziński, Deanna Needell, Elizaveta Rebrova +1
Randomized Kaczmarz methods form a family of linear system solvers which converge by repeatedly projecting their iterates onto randomly sampled equations. While effective in some c…