4 papers · 1 filter
ReTrace: Rejected-Trajectory Conditioning for Speculative Decoding
Luxi Lin, Zhanpeng Zeng, Shuang Peng +2
Speculative decoding accelerates autoregressive language model inference by having a lightweight draft model propose multiple candidate tokens, which are then verified in parallel…
Training-Free Hashing-Based Attention via Binary Principal Components
Daohai Yu, Zhanpeng Zeng, Keyu Chen +6
Long-context large language models (LLMs) are increasingly deployed in real-world applications, yet self-attention remains a major efficiency bottleneck -- especially during decodi…
Empirical Bayes Conformal Prediction for Vision and Language Models
Jiapeng Zeng, Yogesh Prabhu, Zhanpeng Zeng +2
Conformal prediction (CP) gives distribution-free coverage for modern vision and language models, but it is often forced to make a ranking decision from a single unstable nonconfor…
Efficiently Aligning Draft Models via Parameter- and Data-Efficient Adaptation
Luxi Lin, Zhihang Lin, Zhanpeng Zeng +5
Speculative decoding accelerates LLM inference but suffers from performance degradation when target models are fine-tuned for specific domains. A naive solution is to retrain draft…