Speculative Decoding with CTC-based Draft Model for LLM Inference Acceleration

Zhuofan Wen, Shangtong Gui, Yang Feng

Inference acceleration of large language models (LLMs) has been put forward in many application scenarios and speculative decoding has shown its advantage in addressing inference a…