AngelSpec: Towards Real-World High Performance Inference with Speculative Decoding

Hong Liu, Rui Cen, Junhan Shi +10

Speculative decoding accelerates large language model inference without changing the target distribution, but no single drafting structure performs best across real-world workloads…