expert activation cost 1inference efficiency 1large language models 1mixture-of-experts 1speculative decoding 1
From the 1 of 2 linked papers with an AI index.
2 papers
cs.CL2026
Less Experts, Faster Decoding: Cost-Aware Speculative Decoding for Mixture-of-Experts
Jincheng Xie, Runheng Liu, Heyan Huang +4
The paper introduces EcoSpec, a cost-aware speculative decoding method that selects draft tokens to minimize expert activation overhead in large mixture-of-experts language models,…
cs.DC2026
SPECTRE: Hybrid Ordinary-Parallel Speculative Serving for Resource-Efficient LLM Inference
Jincheng Xie, Yawen Ling, Qi Xiao +4
LLM serving platforms are increasingly deployed as multi-model cloud systems, where user demand is often long-tailed: a few popular large models receive most requests, while many s…