Multi-Draft Speculative Sampling: Canonical Decomposition and Theoretical Limits

Ashish Khisti, M. Reza Ebrahimi, Hassan Dbouk +3

We consider multi-draft speculative sampling, where the proposal sequences are sampled independently from different draft models. At each step, a token-level draft selection scheme…