Less Experts, Faster Decoding: Cost-Aware Speculative Decoding for Mixture-of-Experts

Jincheng Xie, Runheng Liu, Heyan Huang +4

Sparse Mixture-of-Experts (MoE) models have become an important approach for scaling Large Language Models (LLMs), but their inference efficiency depends strongly on expert activat…