10 papers
FastEEP: Fast Intra-Instance Elastic Scaling for Expert-Parallel MoE Serving
Yunfei Feng, Jingkai He, Shiqi Liu +3
Large-scale expert parallelism (EP) has become the standard deployment configuration for Mixture-of-Experts (MoE) models in production. Meanwhile, the highly dynamic nature of LLM…
DeltaBox: Scaling Stateful AI Agents with Millisecond-Level Sandbox Checkpoint/Rollback
Yunpeng Dong, Jingkai He, Shiqi Liu +7
LLM-powered AI agents require high-frequency state exploration (e.g., test-time tree search and reinforcement learning), relying on rapid checkpoint and rollback (C/R) of the compl…
Sharpen the Spec, Cut the Code: A Case for Generative File System with SYSSPEC
Qingyuan Liu, Mo Zou, Hengbin Zhang +3
File systems are critical OS components that require constant evolution to support new hardware and emerging application needs. However, the traditional paradigm of developing feat…
MobiAgent: A Systematic Framework for Customizable Mobile Agents
Cheng Zhang, Erhu Feng, Xi Zhao +7
With the rapid advancement of Vision-Language Models (VLMs), GUI-based mobile agents have emerged as a key development direction for intelligent mobile systems. However, existing a…
History Rhymes: Accelerating LLM Reinforcement Learning with RhymeRL
Jingkai He, Tianjian Li, Erhu Feng +5
With the rapid advancement of large language models (LLMs), reinforcement learning (RL) has emerged as a pivotal methodology for enhancing the reasoning capabilities of LLMs. Unlik…
Topology-Aware Virtualization over Inter-Core Connected Neural Processing Units
Dahu Feng, Erhu Feng, Dong Du +4
With the rapid development of artificial intelligence (AI) applications, an emerging class of AI accelerators, termed Inter-core Connected Neural Processing Units (NPU), has been a…
Get Experience from Practice: LLM Agents with Record & Replay
Erhu Feng, Wenbo Zhou, Zibin Liu +8
AI agents, empowered by Large Language Models (LLMs) and communication protocols such as MCP and A2A, have rapidly evolved from simple chatbots to autonomous entities capable of ex…
CHIME: A Case for Efficient Long-Context Attention-FC Disaggregated Inference with DIMM-PIM
Qingyuan Liu, Liyan Chen, Haocheng Wang +6
Attention-FC Disaggregated (AFD) LLM inference systems offload memory-bound Attention operations to memory-rich accelerators (e.g., CPUs, HBM-PIM) while retaining compute-bound Ful…
HeteroPod: XPU-Accelerated Infrastructure Offloading for Commodity Cloud-Native Applications
Bicheng Yang, Jingkai He, Dong Du +2
Cloud-native systems increasingly rely on infrastructure services (e.g., service meshes, monitoring agents), which compete for resources with user applications, degrading performan…
Jiagu: Optimizing Serverless Computing Resource Utilization with Harmonized Efficiency and Practicability
Qingyuan Liu, Yanning Yang, Dong Du +5
Current serverless platforms struggle to optimize resource utilization due to their dynamic and fine-grained nature. Conventional techniques like overcommitment and autoscaling fal…