Adversarial Prompts for Acceptance Collapse in Speculative Decoding

Run Wang, Chaoyi Zhou, Xi Liu +7

Lossless acceleration schemes, such as speculative decoding, promise significant inference speedups by relying on dynamic token-level alignment between a draft and a target model.…