17 papers
Tensor Methods for Language Models: From Token Representation to Training, Adaptation, Inference, Compression, and Interpretability
Matvei Tarasov, Salman Ahmadi-Asl, Andre L. F. de Almeida +1
Large language models (LLMs) are built from structured high-dimensional objects such as token representations, weights, adaptation updates, caches, and activations, whose multiline…
Adaptive Randomized Pivoting for Tensor Cross Approximation in the T-Product Framework
Ahmadsho Akdodshoev, Valentin Leplat, Salman Ahmadi-Asl
This paper studies extensions of adaptive randomized pivoting (ARP), recently introduced for matrix column subset selection, to tensors in the t-product framework. We propose two c…
A New Tensor Network: Tubal Tensor Train and Its Applications
Salman Ahmadi-Asl, Valentin Leplat, Anh-Huy Phan +1
We introduce the tubal tensor train (TTT) decomposition, a tensor-network model that combines the t-product algebra of the tensor singular value decomposition (T-SVD) with the low-…
SAGE: Streaming Agreement-Driven Gradient Sketches for Representative Subset Selection
Ashish Jha, Salman Ahmadi-Asl
Training modern neural networks on large datasets is computationally and energy intensive. We present SAGE, a streaming data-subset selection method that maintains a compact Freque…
Iterative Methods for Computing the Moore-Penrose Pseudoinverse of Quaternion Matrices, with Applications
Valentin Leplat, Salman Ahmadi-Asl, JunJun Pan +1
We develop quaternion--native iterative methods for computing the Moore--Penrose (MP) pseudoinverse of quaternion matrices and analyze their convergence. Our starting point is a da…
Pass-efficient Randomized Algorithms for Low-rank Approximation of Quaternion Matrices
Salman Ahmadi-Asl, Malihe Nobakht Kooshkghazi, Valentin Leplat
Randomized algorithms for low-rank approximation of quaternion matrices have gained increasing attention in recent years. However, existing methods overlook pass efficiency, the ab…