Bastion: Budget-Aware Speculative Decoding with Tree-structured Block Diffusion Drafting

Soowon Oh, Nam Cao, Yujin Kim +4

Block-diffusion drafters have recently emerged as a powerful alternative for speculative decoding by predicting multiple future-token distributions in a single parallel step. Howev…