Token-Driven GammaTune: Adaptive Calibration for Enhanced Speculative Decoding

Aayush Gautam, Susav Shrestha, Narasimha Reddy

Speculative decoding accelerates large language model (LLM) inference by using a smaller draft model to propose tokens, which are then verified by a larger target model. However, s…