7 citations · 16 across the 24 of their papers we have counts for
6 papers · 1 filter
AgentWeave: Efficient Distributed Agent Serving via Flow Decomposition
Kaibin Guo, Pengtu Li, Zicong Hong +4
Agent applications are evolving into workflows composed of large language model (LLM) calls and tool invocations, termed modules. Existing distributed agent systems typically adopt…
TwinQuant: Learnable Subspace Decomposition for 4-Bit LLM Quantization
Haodong Wang, Junjie Liu, Zicong Hong +4
4-bit quantization reduces the memory footprint and latency of large language model inference, but its aggressive precision reduction can severely degrade accuracy. Prior methods a…
DMoE: Dual Routing and Dynamic Scheduling for Efficient On-Device MoE-based LLM Serving
Haodong Wang, Qihua Zhou, Zicong Hong +1
The mixture of experts (MoE) model is a sparse variant of large language models (LLMs), designed to hold a better balance between intelligent capability and computational overhead.…
Mell: Memory-Efficient Large Language Model Serving via Multi-GPU KV Cache Management
Liu Qianli, Hong Zicong, Chen Fahao +2
Serving large language models (LLMs) for massive users is challenged by the significant memory footprint of the transient state, known as the key-value (KV) cache, which scales wit…
Communication-Efficient Sparsely-Activated Model Training via Sequence Migration and Token Condensation
Fahao Chen, Peng Li, Zicong Hong +2
Mixture-of-Experts (MoE) is an emerging technique for scaling large models with sparse activation. MoE models are typically trained in a distributed manner with an expert paralleli…
OTAS: An Elastic Transformer Serving System via Token Adaptation
Jinyu Chen, Wenchao Xu, Zicong Hong +4
Transformer model empowered architectures have become a pillar of cloud services that keeps reshaping our society. However, the dynamic query loads and heterogeneous user requireme…