Showing cs.CLShow all
2 papers · 1 filter
cs.CL2026
GRAFT: Adaptive DLM-Based Draft Tree Construction with Target-Distilled Edge Scoring
Xuming Ye, Zeming Ma, Runjie Yu +5
Tree-based speculative decoding raises the mean accepted tokens of standard speculative decoding by verifying multiple draft paths, and existing tree builders typically construct t…
cs.CL2024
KVPruner: Structural Pruning for Faster and Memory-Efficient Large Language Models
Bo Lv, Quan Zhou, Xuanang Ding +2
The bottleneck associated with the key-value(KV) cache presents a significant challenge during the inference processes of large language models. While depth pruning accelerates inf…