1 paper
Vivien Tran-Thien, Richard Nock
Speculative decoding is a successful technique speeding up inference of a target autoregressive language model via a fast drafter model. Lossy speculative decoding allows a drift w…