SpecRouter: Adaptive Routing for Multi-Level Speculative Decoding in Large Language Models

Hang Wu, Jianian Zhu, Yinghui Li +3

Large Language Models (LLMs) present a critical trade-off between inference quality and computational cost: larger models offer superior capabilities but incur significant latency,…