10 papers
RPLSS: A randomized projected linear systems solver
Meng-Long Xiao, Tao Li, Deanna Needell
The projected linear system solver (PLSS), by incrementally appending columns to a random or deterministic sketching matrix, provides an attractive finite termination property for…
Stochastic Gradient Descent for Incomplete Tensor Linear Systems
Anna Ma, Deanna Needell, Alexander Xue
Solving large tensor linear systems poses significant challenges due to the high volume of data stored, and it only becomes more challenging when some of the data is missing. Recen…
Attention Mechanisms Through the Lens of Numerical Methods: Approximation Methods and Alternative Formulations
Michel Fabrice Serret, Alice Cortinovis, Yijun Dong +10
The attention mechanism is the computational core of modern Transformer architectures, but its quadratic complexity in the input sequence length is the bottleneck for large-scale i…
Are Greedy Task Orderings Better Than Random in Continual Linear Regression?
Matan Tsipory, Ran Levinstein, Itay Evron +3
We analyze task orderings in continual learning for linear regression, assuming joint realizability of training data. We focus on orderings that greedily maximize dissimilarity bet…
Cauchy Random Features for Operator Learning in Sobolev Space
Chunyang Liao, Deanna Needell, Hayden Schaeffer
Operator learning is the approximation of operators between infinite dimensional Banach spaces using machine learning approaches. While most progress in this area has been driven b…
Differentially Private Random Feature Model
Chunyang Liao, Deanna Needell, Hayden Schaeffer +1
Designing privacy-preserving machine learning algorithms has received great attention in recent years, especially in the setting when the data contains sensitive information. Diffe…