4 papers
Democratizing MoE inference on commodity GPUs with CoMoE
Ruwen Fan, Yuezhi Zu, Minhui Xie +6
Deploying Mixture-of-Experts (MoE) models relies heavily on Expert Parallelism, which generates intense inter-GPU communication. Consequently, state-of-the-art inference systems re…
SPORK: Self-Speculative Forking to Accelerate Agentic LLM Inference
Huajun Bai, Weiwei Lv, Huichuan Zheng +2
LLM agents are becoming a common interface for research, coding, and question answering, yet their Thought-Action-Observation loop is often serial: the model reasons, emits a tool…
Efficient Training on Multiple Consumer GPUs with RoundPipe
Yibin Luo, Shiwei Gao, Huichuan Zheng +2
Fine-tuning Large Language Models (LLMs) on consumer-grade GPUs is highly cost-effective, yet constrained by limited GPU memory and slow PCIe interconnects. Pipeline parallelism co…
SwitchDelta: Asynchronous Metadata Updating for Distributed Storage with In-Network Data Visibility
Junru Li, Qing Wang, Zhe Yang +3
Distributed storage systems typically maintain strong consistency between data nodes and metadata nodes by adopting ordered writes: 1) first installing data; 2) then updating metad…