5 papers
Numerical stability analysis of large language models
Stanislav Budzinskiy, Wenyi Fang, Longbin Zeng +1
Transformers are the state-of-the-art architecture for large language models, and a key to their scalability is the strategic usage of low-precision arithmetic. We develop a mixed-…
LAMP: Look-Ahead Mixed-Precision Inference of Large Language Models
Stanislav Budzinskiy, Marian Gloser, Tolunay Yilmaz +5
Mixed-precision computations are a hallmark of the current stage of AI, driving the progress in large language models towards efficient, locally deployable solutions. This article…
Low-rank cross approximation of function-valued tensors for reduced-order modeling of parametric PDEs
Stanislav Budzinskiy, Vladimir Kazeev, Maxim Olshanskii
The paper considers function-valued tensors, viewed as multidimensional arrays with entries in an abstract Hilbert space. Despite the absence of the algebraic structure of a field,…
When big data actually are low-rank, or entrywise approximation of certain function-generated matrices
Stanislav Budzinskiy
The article concerns low-rank approximation of matrices generated by sampling a smooth function of two -dimensional variables. We identify several misconceptions surrounding a c…
Entrywise tensor-train approximation of large tensors via random embeddings
Stanislav Budzinskiy
The theory of low-rank tensor-train approximation is well understood when the approximation error is measured in the Frobenius norm. The entrywise maximum norm is equally important…