4 papers
Randomized Tucker-Sketched GMRES
Alberto Bucci, Martina Iannacito, Mirjeta Pasha +1
We address the problem of solving large-scale tensor-structured linear systems in the Tucker format. In this setting, standard iterative solvers such as GMRES face a fundamental bo…
Attention Mechanisms Through the Lens of Numerical Methods: Approximation Methods and Alternative Formulations
Michel Fabrice Serret, Alice Cortinovis, Yijun Dong +10
The attention mechanism is the computational core of modern Transformer architectures, but its quadratic complexity in the input sequence length is the bottleneck for large-scale i…
Efficient Sketching-Based Summation of Tucker Tensors
Rudi Smith, Mirjeta Pasha, Andrés Galindo-Olarte +5
We present efficient, sketching-based methods for the summation of tensors in Tucker format. Leveraging the algebraic structure of Khatri-Rao and Kronecker products, our approach e…
A tangential low-rank ADI method for solving indefinite Lyapunov equations
Rudi Smith, Steffen W. R. Werner
Continuous-time algebraic Lyapunov equations have become an essential tool in various applications. In the case of large-scale sparse coefficient matrices and indefinite constant t…