1 citations · 1 across the 9 of their papers we have counts for
12 papers
Quantile Randomized Kaczmarz for Streaming Linear Systems with Massart Noise
Emeric Battaglia, Jian-Feng Cai, Junren Chen +3
Quantile randomized Kaczmarz (QRK) has proven to be an efficient solver for corrupted linear systems and has received much attention. It was recently shown by Cai et al. (SIAM J. M…
RPLSS: A randomized projected linear systems solver
Meng-Long Xiao, Tao Li, Deanna Needell
The projected linear system solver (PLSS), by incrementally appending columns to a random or deterministic sketching matrix, provides an attractive finite termination property for…
Attention Mechanisms Through the Lens of Numerical Methods: Approximation Methods and Alternative Formulations
Michel Fabrice Serret, Alice Cortinovis, Yijun Dong +10
The attention mechanism is the computational core of modern Transformer architectures, but its quadratic complexity in the input sequence length is the bottleneck for large-scale i…
Are Greedy Task Orderings Better Than Random in Continual Linear Regression?
Matan Tsipory, Ran Levinstein, Itay Evron +3
We analyze task orderings in continual learning for linear regression, assuming joint realizability of training data. We focus on orderings that greedily maximize dissimilarity bet…
Stochastic Gradient Descent for Incomplete Tensor Linear Systems
Anna Ma, Deanna Needell, Alexander Xue
Solving large tensor linear systems poses significant challenges due to the high volume of data stored, and it only becomes more challenging when some of the data is missing. Recen…
A federated Kaczmarz algorithm
Halyun Jeong, Deanna Needell, Chi-Hao Wu
In this paper, we propose a federated algorithm for solving large linear systems that is inspired by the classic randomized Kaczmarz algorithm. We provide convergence guarantees of…
Cauchy Random Features for Operator Learning in Sobolev Space
Chunyang Liao, Deanna Needell, Hayden Schaeffer
Operator learning is the approximation of operators between infinite dimensional Banach spaces using machine learning approaches. While most progress in this area has been driven b…
Randomized Kaczmarz Methods with Beyond-Krylov Convergence
Michał Dereziński, Deanna Needell, Elizaveta Rebrova +1
Randomized Kaczmarz methods form a family of linear system solvers which converge by repeatedly projecting their iterates onto randomly sampled equations. While effective in some c…
Differentially Private Random Feature Model
Chunyang Liao, Deanna Needell, Hayden Schaeffer +1
Designing privacy-preserving machine learning algorithms has received great attention in recent years, especially in the setting when the data contains sensitive information. Diffe…
Convergence of Manifold Filter-Combine Networks
David R. Johnson, Joyce Chew, Siddharth Viswanath +4
In order to better understand manifold neural networks (MNNs), we introduce Manifold Filter-Combine Networks (MFCNs). The filter-combine framework parallels the popular aggregate-c…
Fine-grained Analysis and Faster Algorithms for Iteratively Solving Linear Systems
Michał Dereziński, Daniel LeJeune, Deanna Needell +1
Despite being a key bottleneck in many machine learning tasks, the cost of solving large linear systems has proven challenging to quantify due to problem-dependent quantities such…
Stochastic gradient descent for streaming linear and rectified linear systems with adversarial corruptions
Halyun Jeong, Deanna Needell, Elizaveta Rebrova
We propose SGD-exp, a stochastic gradient descent approach for linear and ReLU regressions under Massart noise (adversarial semi-random corruption model) for the fully streaming se…