2.7k citations · 2.7k across the 17 of their papers we have counts for
1 paper · 2 filters
Amirhossein Bozorgkhoo, Igor Molybog
Speculative decoding is a technique that uses multiple language models to accelerate infer- ence. Previous works have used an experi- mental approach to optimize the throughput of…