PolyGraph: Plan-Buffered CUDA Graphs for Adaptive Speculative Decoding

Junchao Yang

Adaptive speculative decoding changes the active batch size B and draft length d from one decode step to the next, whereas a CUDA Graph replays a fixed launch configuration. The re…