4 papers · 1 filter
ExpertPlex: A High-Goodput Disaggregated Serving System for MoE LLMs with Adaptive Persistent Kernels
Bingyang Wu, Chao Jin, Zili Zhang +6
LLMs scale Mixture-of-Experts (MoE) parameters for superior intelligence, but massive weights and dynamic computation impede efficient serving. Existing instance-level prefill-deco…
UltraEP: Unleash MoE Training and Inference on Rack-Scale Nodes with Near-Optimal Load Balancing
Xinming Wei, Chao Jin, Tuo Dai +10
Large-scale expert parallelism (EP) is becoming pivotal for training and serving frontier MoE models, but it also amplifies device-level expert load imbalance into compute straggle…
Agent.xpu: Efficient Scheduling of Agentic LLM Workloads on Heterogeneous SoC
Xinming Wei, Jiahao Zhang, Haoran Li +6
Personal LLM agents increasingly combine foreground reactive interactions with background proactive monitoring, forming long-lived, stateful LLM flows that interleave prefill and t…
ACE-GNN: Adaptive GNN Co-Inference with System-Aware Scheduling in Dynamic Edge Environments
Ao Zhou, Jianlei Yang, Tong Qiao +5
The device-edge co-inference paradigm effectively bridges the gap between the high resource demands of Graph Neural Networks (GNNs) and limited device resources, making it a promis…