adaptive pruning 1batch processing 1inference acceleration 1large language models 1speculative decoding 1
From the 1 of 4 linked papers with an AI index.
Showing cs.CLShow all
2 papers · 1 filter
cs.CL2026
AngelSpec: Towards Real-World High Performance Inference with Speculative Decoding
Hong Liu, Rui Cen, Junhan Shi +10
Speculative decoding accelerates large language model inference without changing the target distribution, but no single drafting structure performs best across real-world workloads…
cs.CL2026
D-cut: Adaptive Verification Depth Pruning for Batched Speculative Decoding
Tianyu Liu, Yuhao Shen, Rui Cen +7
The paper introduces D-Cut, an adaptive method that prunes draft tokens across a batch to focus verification on the most promising tokens, improving the speed of speculative decodi…