TALON: Confidence-Aware Speculative Decoding with Adaptive Token Trees

Tianyu Liu, Qitan Lv, Yuhao Shen +2

Speculative decoding (SD) has become a standard technique for accelerating LLM inference without sacrificing output quality. Recent advances in speculative decoding have shifted fr…