2 citations · 5 across the 7 of their papers we have counts for
1 paper · 1 filter
Tanishq Kumar, Tri Dao, Avner May
Autoregressive decoding is bottlenecked by its sequential nature. Speculative decoding has become a standard way to accelerate inference by using a fast draft model to predict upco…