15 papers
MPrune: Hierarchical Collaborative Pruning for Efficient Multi-Modal Multi-Agent Retrieval-Augmented Generation
Taolin Zhang, Weizi shao, Zijie Zhou +5
Recent advances in multi-modal retrieval-augmented generation (mRAG), which augments multi-modal large language models (MLLMs) with external knowledge, have shown that collective i…
Efficiency and Cost Alignment in Batched LLM Serving via Resource-Fair Scheduling
Dayi Yao, Zijie Zhou
This paper studies a resource-allocation inefficiency in batched large language model (LLM) serving: heterogeneous requests that share a decode batch impose max-driven computationa…
Online Linear Programming for Multi-Objective Routing in LLM Serving
Zixi Chen, Yinyu Ye, Zijie Zhou
We study the online routing problem in large language model serving, where requests arrive sequentially and must be dispatched to parallel decode workers under tight batch-size and…
Geometry-Aware Online Scheduling for LLM Serving: From Theoretical Bound to System Practice
Li Kong, Qi Qi, Yinyu Ye +1
The rapid growth of interactive Large Language Model serving has made efficient management of dynamic Key-Value cache footprints increasingly important for inference performance. M…
Hyperbolic and Evidence-Prioritized Experts for Large Vision-Language Models
Zijie Zhou, Dandan Zhu, Hangxiangpan Wang +3
Large Vision-Language Models (LVLMs) have demonstrated impressive performance on multimodal tasks through scaled architectures and extensive training. Recent studies introduce Mixt…
AutoMCU: Feasibility-First MCU Neural Network Customization via LLM-based Multi-Agent Systems
Penglin Dai, Zijie Zhou, Xincao Xu +3
Deploying neural networks on microcontroller units (MCUs) is critical for edge intelligence but remains challenging due to tight memory, storage, and computation constraints. Exist…
Learning Transferable Topology Priors for Multi-Agent LLM Collaboration Across Domains
Taolin Zhang, Zijie Zhou, Jiuheng Wan +4
Large language model (LLM)-based multi-agent systems have shown strong potential for complex reasoning by coordinating specialized agents through structured communication. However,…
Tackling the Data-Parallel Load Balancing Bottleneck in LLM Serving: Practical Online Routing at Scale
Tianci Bu, Yuan Lyu, Zixi Chen +6
Data-parallel (DP) load balancing has emerged as a first-order bottleneck in large-scale LLM serving. When a model is sharded across devices via tensor parallelism (TP) or expert p…
A Queueing-Theoretic Framework for Stability Analysis of LLM Inference with KV Cache Memory Constraints
Chengyi Nie, Nian Si, Zijie Zhou
The rapid adoption of large language models (LLMs) has created significant challenges for efficient inference at scale. Unlike traditional workloads, LLM inference is constrained b…
Position: LLM Serving Needs Mathematical Optimization and Algorithmic Foundations, Not Just Heuristics
Zijie Zhou
This position paper argues that LLM inference serving has outgrown generic heuristics and now demands mathematical optimization and algorithmic foundations. Despite rapid advances…
PolarMem: A Training-Free Polarized Latent Graph Memory for Verifiable Vision-Language Models
Zhisheng Chen, Tingyu Wu, Zijie Zhou +7
Memory is not merely a storage mechanism for intelligent systems, but a structure for organizing evidence and constraining belief. This is especially important for multimodal reaso…
Load Balancing with Evolving Workloads in Large-Scale Language Model Serving
Zixi Chen, Tianci Bu, Chendong Song +3
Serving large language models requires assigning requests whose workloads grow during processing and whose completion times are unknown. With a synchronization barrier, lightly loa…
Analytical Provisioning for Attention-FFN Disaggregated LLM Serving under Stochastic Workloads
Chendong Song, Meixuan Wang, Hang Zhou +5
Attentio-FFN disaggregation (AFD) is an emerging architecture for LLM decoding that separates state-heavy, KV-cache-dominated Attention computation from stateless, compute-intensiv…
Adaptively Robust LLM Inference Optimization under Prediction Uncertainty
Zixi Chen, Yinyu Ye, Zijie Zhou
We study the problem of optimizing Large Language Model (LLM) inference scheduling to minimize total latency. LLM inference is an online and multi-task service process and also hea…
LLM Serving Optimization with Variable Prefill and Decode Lengths
Meixuan Wang, Yinyu Ye, Zijie Zhou
We study offline scheduling for large language model (LLM) serving under a fixed KV-cache memory budget, where requests have heterogeneous prompt (prefill) and response (decode) le…