8 citations · 15 across the 7 of their papers we have counts for
3 papers · 1 filter
LampAttention: Look-Ahead Mixed-Precision FlashAttention for Dedicated Accelerators
Stanislav Budzinskiy, Marian Gloser, Tolunay Yilmaz +5
While most attention logits can be computed in low precision without degrading numerical stability, current attention kernels fail to exploit this phenomenon. We introduce a novel…
LAMP: Look-Ahead Mixed-Precision Inference of Large Language Models
Stanislav Budzinskiy, Marian Gloser, Tolunay Yilmaz +5
Mixed-precision computations are a hallmark of the current stage of AI, driving the progress in large language models towards efficient, locally deployable solutions. This article…
Metis: Training LLMs with FP4 Quantization
Hengjie Cao, Mengyi Chen, Yifeng Yang +13
This work identifies anisotropy in the singular value spectra of parameters, activations, and gradients as the fundamental barrier to low-bit training of large language models (LLM…