Dustin: Draft-Augmented Sparse Verification for Efficient Long-Context Generation with Speculative Decoding

WenHung Lee, Jian-Jia Chen, Xiaolin Lin +6

While speculative decoding improves inference throughput for multi-batch long-context Large Language Models (LLMs), its efficiency is often limited by a verification bottleneck whe…