4 papers
LampAttention: Look-Ahead Mixed-Precision FlashAttention for Dedicated Accelerators
Stanislav Budzinskiy, Marian Gloser, Tolunay Yilmaz +5
While most attention logits can be computed in low precision without degrading numerical stability, current attention kernels fail to exploit this phenomenon. We introduce a novel…
Adaptivity Under Realizability Constraints: Comparing In-Context and Agentic Learning
Anastasis Kratsios, A. Martina Neuman, Philipp Petersen
We compare in-context learning with fixed queries and agentic learning with adaptive queries for uniform approximation of task families. We consider two settings: an unrestricted r…
LAMP: Look-Ahead Mixed-Precision Inference of Large Language Models
Stanislav Budzinskiy, Marian Gloser, Tolunay Yilmaz +5
Mixed-precision computations are a hallmark of the current stage of AI, driving the progress in large language models towards efficient, locally deployable solutions. This article…
Numerical stability analysis of large language models
Stanislav Budzinskiy, Wenyi Fang, Longbin Zeng +1
Transformers are the state-of-the-art architecture for large language models, and a key to their scalability is the strategic usage of low-precision arithmetic. We develop a mixed-…