3 papers
cs.CV2025
SpecVLM: Fast Speculative Decoding in Vision-Language Models
Haiduo Huang, Fuwei Yang, Zhenhua Liu +4
Speculative decoding is a powerful way to accelerate autoregressive large language models (LLMs), but directly porting it to vision-language models (VLMs) faces unique systems cons…
cs.CL2025
Jakiro: Boosting Speculative Decoding with Decoupled Multi-Head via MoE
Haiduo Huang, Fuwei Yang, Zhenhua Liu +7
Speculative decoding (SD) accelerates large language model inference by using a smaller draft model to predict multiple tokens, which are then verified in parallel by the larger ta…
cs.CV2025
Nearly Lossless Adaptive Bit Switching
Haiduo Huang, Zhenhua Liu, Tian Xia +2
Model quantization is widely applied for compressing and accelerating deep neural networks (DNNs). However, conventional Quantization-Aware Training (QAT) focuses on training DNNs…