15 papers
MPrune: Hierarchical Collaborative Pruning for Efficient Multi-Modal Multi-Agent Retrieval-Augmented Generation
Taolin Zhang, Weizi shao, Zijie Zhou +5
Recent advances in multi-modal retrieval-augmented generation (mRAG), which augments multi-modal large language models (MLLMs) with external knowledge, have shown that collective i…
Efficiency and Cost Alignment in Batched LLM Serving via Resource-Fair Scheduling
Dayi Yao, Zijie Zhou
This paper studies a resource-allocation inefficiency in batched large language model (LLM) serving: heterogeneous requests that share a decode batch impose max-driven computationa…
Online Linear Programming for Multi-Objective Routing in LLM Serving
Zixi Chen, Yinyu Ye, Zijie Zhou
We study the online routing problem in large language model serving, where requests arrive sequentially and must be dispatched to parallel decode workers under tight batch-size and…
LLM Serving Optimization with Variable Prefill and Decode Lengths
Meixuan Wang, Yinyu Ye, Zijie Zhou
We study offline scheduling for large language model (LLM) serving under a fixed KV-cache memory budget, where requests have heterogeneous prompt (prefill) and response (decode) le…
Geometry-Aware Online Scheduling for LLM Serving: From Theoretical Bound to System Practice
Li Kong, Qi Qi, Yinyu Ye +1
The explosive demand for interactive Large Language Model serving has highlighted the management of the Key-Value cache's dynamic memory footprint as a critical area for performanc…
PolarMem: A Training-Free Polarized Latent Graph Memory for Verifiable Vision-Language Models
Zhisheng Chen, Tingyu Wu, Zijie Zhou +7
Memory is not merely a storage mechanism for intelligent systems, but a structure for organizing evidence and constraining belief. This is especially important for multimodal reaso…