Inference with Reference: Lossless Acceleration of Large Language Models

Nan Yang, Tao Ge, Liang Wang +5

We propose LLMA, an LLM accelerator to losslessly speed up Large Language Model (LLM) inference with references. LLMA is motivated by the observation that there are abundant identi…