5 papers
Beyond Uniform Experts: Cost-Aware Expert Execution for Efficient Multi-Device MoE Inference
Hui Zang, Pengfei Xia, Hong Liu +5
Mixture-of-Experts (MoE) architectures enable language models to achieve unprecedented scale via sparse activation. However, their inference performance is often limited by data mo…
FlexNPU: Transparent NPU Virtualization for Dynamic LLM Prefill-Decode Co-location
Jiongjiong Gu, Jianfeng Wang, Zidong Han +19
Modern AI serving increasingly relies on NPUs for conventional inference and large language model serving. However, current NPU deployments commonly expose physical devices directl…
When KV Cache Reuse Fails in Multi-Agent Systems: Cross-Candidate Interaction is Crucial for LLM Judges
Sichu Liang, Zhenglin Wang, Jiajia Chu +3
Multi-agent LLM systems routinely generate multiple candidate responses that are aggregated by an LLM judge. To reduce the dominant prefill cost in such pipelines, recent work advo…
GAIR: GUI Automation via Information-Joint Reasoning and Group Reflection
Zishu Wei, Qixiang Ma, Xavier Hu +6
Building AI systems for GUI automation task has attracted remarkable research efforts, where MLLMs are leveraged for processing user requirements and give operations. However, GUI…
Pangu Pro MoE: Mixture of Grouped Experts for Efficient Sparsity
Yehui Tang, Xiaosong Li, Fangcheng Liu +19
The surgence of Mixture of Experts (MoE) in Large Language Models promises a small price of execution cost for a much larger model parameter count and learning capacity, because on…