7 citations · 16 across the 24 of their papers we have counts for
28 papers
AgentWeave: Efficient Distributed Agent Serving via Flow Decomposition
Kaibin Guo, Pengtu Li, Zicong Hong +4
Agent applications are evolving into workflows composed of large language model (LLM) calls and tool invocations, termed modules. Existing distributed agent systems typically adopt…
SteerQuant: Steering Quantization Error with Action-Guided Scaling in World-Action Models
Yunhan Wang, Haodong Wang, Zhiming Liu +7
World-action models (WAMs) jointly generate future world states and actions through iterative denoising, using shared weights to process heterogeneous semantic streams of video, pr…
Sparse-WAM: Accelerating World Action Models via Action-Guided Sparse Imagination
Xinling Xie, Haodong Wang, Jiazhi Mi +8
World-action models (WAMs) leverage pretrained video models to improve generalization in robot control by jointly predicting future visual states and actions. This capability comes…
OnlineWM: Causality-Aware Active Online Learning for Effective World Modeling
Yikun Miao, Fangqi Zhu, Quanxin Shou +6
Generative world models aim to predict future states conditioned on actions, where action controllability is fundamental for reliable dynamics modeling. While recent efforts levera…
RedFlow: Redirect Failure into Action-level Corrections for Flow-matching VLA Policy
Zhengyang Yan, Junhao Li, Fangqi Zhu +6
Reinforcement learning (RL) can improve Vision-Language-Action (VLA) policies from deployment experience, but reward- and preference-based RL primarily identifies desirable behavio…
NASDAQ: Normalized Observation Space Dynamics-Augmented Q-Learning
Xinwei Liu, Junyuan Liang, Zicong Hong +2
Augmenting model-free reinforcement learning (RL) with representations learned through observation dynamics prediction (observation-predictive RL) can improve sample efficiency and…
MosaicQuant: Inlier-Outlier Disaggregation for Unified 4-Bit LLM Quantization
Yangjia Hu, Haodong Wang, Zicong Hong +8
4-bit quantization significantly reduces the memory footprint and accelerates the inference of large language models (LLMs). However, its limited bit-width representation struggles…
Director: Accelerating Distributed MoE Serving via Online Proactive Expert Placement
Qianli Liu, Kaibin Guo, Zicong Hong +5
Expert parallelism has become the prevailing paradigm to serve Mixture-of-Experts (MoE) models. Its efficiency depends on the communication and computation latencies of the GPUs, w…
TwinQuant: Learnable Subspace Decomposition for 4-Bit LLM Quantization
Haodong Wang, Junjie Liu, Zicong Hong +4
4-bit quantization reduces the memory footprint and latency of large language model inference, but its aggressive precision reduction can severely degrade accuracy. Prior methods a…
KVDrive: A Holistic Multi-Tier KV Cache Management System for Long-Context LLM Inference
Jian Lin, Jiazhi Mi, Zicong Hong +5
Supporting long-context LLMs is challenging due to the substantial memory demands of the key-value (KV) cache. Existing offloading systems store the full cache in host memory and s…
PPAI: Enabling Personalized LLM Agent Interoperability for Collaborative Edge Intelligence
Zile Wang, Qianli Liu, Kaibin Guo +4
Deploying large language model (LLM) on edge device enables personalized LLM agents for various users. The growing availability of diverse personalized agents presents a unique opp…
HALO: A Unified Vision-Language-Action Model for Embodied Multimodal Chain-of-Thought Reasoning
Quanxin Shou, Fangqi Zhu, Shawn Chen +9
Vision-Language-Action (VLA) models have shown strong performance in robotic manipulation, but often struggle in long-horizon or out-of-distribution scenarios due to the lack of ex…
WMPO: World Model-based Policy Optimization for Vision-Language-Action Models
Fangqi Zhu, Zhengyang Yan, Zicong Hong +3
Vision-Language-Action (VLA) models have shown strong potential for general-purpose robotic manipulation, but their reliance on expert demonstrations limits their ability to learn…
DiEP: Adaptive Mixture-of-Experts Compression through Differentiable Expert Pruning
Sikai Bai, Haoxi Li, Jie Zhang +2
Despite the significant breakthrough of Mixture-of-Experts (MoE), the increasing scale of these MoE models presents huge memory and storage challenges. Existing MoE pruning methods…
Krul: Efficient State Restoration for Multi-turn Conversations with Dynamic Cross-layer KV Sharing
Junyi Wen, Junyuan Liang, Zicong Hong +3
Efficient state restoration in multi-turn conversations with large language models (LLMs) remains a critical challenge, primarily due to the overhead of recomputing or loading full…
DMoE: Dual Routing and Dynamic Scheduling for Efficient On-Device MoE-based LLM Serving
Haodong Wang, Qihua Zhou, Zicong Hong +1
The mixture of experts (MoE) model is a sparse variant of large language models (LLMs), designed to hold a better balance between intelligent capability and computational overhead.…
The MoE-Empowered Edge LLMs Deployment: Architecture, Challenges, and Opportunities
Ning Li, Song Guo, Tuo Zhang +5
The powerfulness of LLMs indicates that deploying various LLMs with different scales and architectures on end, edge, and cloud to satisfy different requirements and adaptive hetero…
Klotski: Efficient Mixture-of-Expert Inference via Expert-Aware Multi-Batch Pipeline
Zhiyuan Fang, Yuegui Huang, Zicong Hong +5
Mixture of Experts (MoE), with its distinctive sparse structure, enables the scaling of language models up to trillions of parameters without significantly increasing computational…
Fate: Fast Edge Inference of Mixture-of-Experts Models via Cross-Layer Gate
Zhiyuan Fang, Zicong Hong, Yuegui Huang +5
Large Language Models (LLMs) have demonstrated impressive performance across various tasks, and their application in edge scenarios has attracted significant attention. However, sp…
Mell: Memory-Efficient Large Language Model Serving via Multi-GPU KV Cache Management
Liu Qianli, Hong Zicong, Chen Fahao +2
Serving large language models (LLMs) for massive users is challenged by the significant memory footprint of the transient state, known as the key-value (KV) cache, which scales wit…
Communication-Efficient Sparsely-Activated Model Training via Sequence Migration and Token Condensation
Fahao Chen, Peng Li, Zicong Hong +2
Mixture-of-Experts (MoE) is an emerging technique for scaling large models with sparse activation. MoE models are typically trained in a distributed manner with an expert paralleli…
GriDB: Scaling Blockchain Database via Sharding and Off-Chain Cross-Shard Mechanism
Zicong Hong, Song Guo, Enyuan Zhou +3
Blockchain databases have attracted widespread attention but suffer from poor scalability due to underlying non-scalable blockchains. While blockchain sharding is necessary for a s…
OTAS: An Elastic Transformer Serving System via Token Adaptation
Jinyu Chen, Wenchao Xu, Zicong Hong +4
Transformer model empowered architectures have become a pillar of cloud services that keeps reshaping our society. However, the dynamic query loads and heterogeneous user requireme…
Training and Serving System of Foundation Models: A Comprehensive Survey
Jiahang Zhou, Yanyu Chen, Zicong Hong +6
Foundation models (e.g., ChatGPT, DALL-E, PengCheng Mind, PanGu-) have demonstrated extraordinary performance in key technological areas, such as natural language processing and…