EVA: Accelerating LLM Decoding via an Efficient Vector Quantization Architecture

Bowen Duan, Cong Guo, Chiyue Wei +8

Large Language Models (LLMs) have achieved impressive performance across diverse domains but remain inefficient during the autoregressive decoding phase. Unlike the prefill stage,…