3 papers
cs.AR2025
From Quarter to All: Accelerating Speculative LLM Decoding via Floating-Point Exponent Remapping and Parameter Sharing
Yushu Zhao, Yubin Qin, Yang Wang +5
Large language models achieve impressive performance across diverse tasks but exhibit high inference latency due to their large parameter sizes. While quantization reduces model si…
cs.CL2025
MoBiLE: Efficient Mixture-of-Experts Inference on Consumer GPU with Mixture of Big Little Experts
Yushu Zhao, Yubin Qin, Yang Wang +5
Mixture-of-Experts (MoE) models have recently demonstrated exceptional performance across a diverse range of applications. The principle of sparse activation in MoE models facilita…
cs.LG2024
Catch-Up Distillation: You Only Need to Train Once for Accelerating Sampling
Shitong Shao, Xu Dai, Lujun Li +3
Diffusion Probability Models (DPMs) have made impressive advancements in various machine learning domains. However, achieving high-quality synthetic samples typically involves perf…