Mistletoe: Stealthy Acceleration-Collapse Attacks on Speculative Decoding

Shuoyang Sun, Chang Dai, Hao Fang +6

Speculative decoding has become a widely adopted technique for accelerating large language model (LLM) inference by drafting multiple candidate tokens and verifying them with a tar…