Showing 2026Show all
3 papers · 1 filter
cs.CL2026
TreeGraft: Adaptive Multi-Drafter Grafting for Tree-Based Speculative Decoding
Jiaming Fan, Daming Cao, Canchen Huang +6
Speculative decoding accelerates large language model inference through a draft-then-verify paradigm. Building on this, tree-structured methods improve inference by organizing prop…
cs.LG2026
Rethinking LLM Ensembling from the Perspective of Mixture Models
Jiale Fu, Yuchu Jiang, Peijun Wu +3
Model ensembling is a well-established technique for improving the performance of machine learning models. Conventionally, this involves averaging the output distributions of multi…
cs.CL2026
Flatter Tokens are More Valuable for Speculative Draft Model Training
Jiaming Fan, Daming Cao, Xiangzhong Luo +3
Speculative Decoding (SD) is a key technique for accelerating Large Language Model (LLM) inference, but it typically requires training a draft model on a large dataset. We approach…