Tensor Methods for Language Models: From Token Representation to Training, Adaptation, Inference, Compression, and Interpretability

Matvei Tarasov, Salman Ahmadi-Asl, Andre L. F. de Almeida +1

Large language models (LLMs) are built from structured high-dimensional objects such as token representations, weights, adaptation updates, caches, and activations, whose multiline…