Amphista: Bi-directional Multi-head Decoding for Accelerating LLM Inference

Zeping Li, Xinlong Yang, Ziheng Gao +7

Large Language Models (LLMs) inherently use autoregressive decoding, which lacks parallelism in inference and results in significantly slow inference speed. While methods such as M…