5 papers
UBTree: Parallel Tree Drafting via Unigram and Bigram Models for Speculative Decoding
Chumeng Liang, Linxuan Wang, Xinyu Peng +6
Speculative decoding accelerates language model inference by verifying multiple draft tokens in a single target-model pass. Recent parallel drafters have achieved breakthrough perf…
DIVERSED: Relaxed Speculative Decoding via Dynamic Ensemble Verification
Ziyi Wang, Siva Rajesh Kasa, Ankith M S +6
Speculative decoding is an effective technique for accelerating large language model inference by drafting multiple tokens in parallel. In practice, its speedup is often bottleneck…
Generalized Discrete Diffusion with Self-Correction
Linxuan Wang, Ziyi Wang, Yikun Bai +3
Self-correction is an effective technique for maintaining parallel sampling in discrete diffusion models with minimal performance degradation. Prior work has explored self-correcti…
SQS: Bayesian DNN Compression through Sparse Quantized Sub-distributions
Ziyi Wang, Nan Jiang, Guang Lin +1
Compressing large-scale neural networks is essential for deploying models on resource-constrained devices. Most existing methods adopt weight pruning or low-bit quantization indivi…
Enhancing Low-Precision Sampling via Stochastic Gradient Hamiltonian Monte Carlo
Ziyi Wang, Yujie Chen, Qifan Song +1
Low-precision training has emerged as a promising low-cost technique to enhance the training efficiency of deep neural networks without sacrificing much accuracy. Its Bayesian coun…