1 citations · 1 across the 11 of their papers we have counts for
5 papers · 1 filter
Quantile-based Loss Filtering for Outlier-Robust Stochastic Gradient Descent
Jamie Haddock, Anna Ma, Elizaveta Rebrova
We study loss-based filtering for finite-sum optimization with a subset of corrupted component functions whose gradients may be highly unreliable. Motivated by minimum-loss-based S…
Inference for Newton Methods with Accelerated Sketch-and-Project via Random Scaling
Xinchen Du, Elizaveta Rebrova, Michał Dereziński +1
We study an online sketched Newton method that approximates the Newton direction at each step via a state-of-the-art sketching solver, called the generalized accelerated sketch-and…
Towards Universal Convergence of Backward Error in Linear System Solvers
Michał Dereziński, Yuji Nakatsukasa, Elizaveta Rebrova
The quest for an algorithm that solves an linear system in time complexity, or when solving up to relative error, is a long-stand…
Attention Mechanisms Through the Lens of Numerical Methods: Approximation Methods and Alternative Formulations
Michel Fabrice Serret, Alice Cortinovis, Yijun Dong +10
The attention mechanism is the computational core of modern Transformer architectures, but its quadratic complexity in the input sequence length is the bottleneck for large-scale i…
Quantile Randomized Kaczmarz Algorithm with Whitelist Trust Mechanism
Sofiia Shvaiko, Longxiu Huang, Elizaveta Rebrova
Randomized Kaczmarz (RK) is a simple and fast solver for consistent overdetermined systems, but it is known to be fragile under noise. We study overdetermined linear sy…