SpecHub: Provable Acceleration to Multi-Draft Speculative Decoding

Ryan Sun, Tianyi Zhou, Xun Chen +1

Large Language Models (LLMs) have become essential in advancing natural language processing (NLP) tasks, but their sequential token generation limits inference speed. Multi-Draft S…