PRISM: Parametrically Refactoring Inference for Speculative Sampling Draft Models

Xuliang Wang, Yuetao Chen, Maochan Zhen +5

Large Language Models (LLMs), constrained by their auto-regressive nature, suffer from slow decoding. Speculative decoding methods have emerged as a promising solution to accelerat…