1 citations · 1 across the 5 of their papers we have counts for
5 papers
MosaicQuant: Inlier-Outlier Disaggregation for Unified 4-Bit LLM Quantization
Yangjia Hu, Haodong Wang, Zicong Hong +8
4-bit quantization significantly reduces the memory footprint and accelerates the inference of large language models (LLMs). However, its limited bit-width representation struggles…
Director: Accelerating Distributed MoE Serving via Online Proactive Expert Placement
Qianli Liu, Kaibin Guo, Zicong Hong +5
Expert parallelism has become the prevailing paradigm to serve Mixture-of-Experts (MoE) models. Its efficiency depends on the communication and computation latencies of the GPUs, w…
KVDrive: A Holistic Multi-Tier KV Cache Management System for Long-Context LLM Inference
Jian Lin, Jiazhi Mi, Zicong Hong +5
Supporting long-context LLMs is challenging due to the substantial memory demands of the key-value (KV) cache. Existing offloading systems store the full cache in host memory and s…
PPAI: Enabling Personalized LLM Agent Interoperability for Collaborative Edge Intelligence
Zile Wang, Qianli Liu, Kaibin Guo +4
Deploying large language model (LLM) on edge device enables personalized LLM agents for various users. The growing availability of diverse personalized agents presents a unique opp…
Intelligence-Endogenous Management Platform for Computing and Network Convergence
Zicong Hong, Xiaoyu Qiu, Jian Lin +5
Massive emerging applications are driving demand for the ubiquitous deployment of computing power today. This trend not only spurs the recent popularity of the \emph{Computing and…