15 papers
MPrune: Hierarchical Collaborative Pruning for Efficient Multi-Modal Multi-Agent Retrieval-Augmented Generation
Taolin Zhang, Weizi shao, Zijie Zhou +5
Recent advances in multi-modal retrieval-augmented generation (mRAG), which augments multi-modal large language models (MLLMs) with external knowledge, have shown that collective i…
Efficiency and Cost Alignment in Batched LLM Serving via Resource-Fair Scheduling
Dayi Yao, Zijie Zhou
This paper studies a resource-allocation inefficiency in batched large language model (LLM) serving: heterogeneous requests that share a decode batch impose max-driven computationa…
Online Linear Programming for Multi-Objective Routing in LLM Serving
Zixi Chen, Yinyu Ye, Zijie Zhou
We study the online routing problem in large language model serving, where requests arrive sequentially and must be dispatched to parallel decode workers under tight batch-size and…
Geometry-Aware Online Scheduling for LLM Serving: From Theoretical Bound to System Practice
Li Kong, Qi Qi, Yinyu Ye +1
The explosive demand for interactive Large Language Model serving has highlighted the management of the Key-Value cache's dynamic memory footprint as a critical area for performanc…
Hyperbolic and Evidence-Prioritized Experts for Large Vision-Language Models
Zijie Zhou, Dandan Zhu, Hangxiangpan Wang +3
Large Vision-Language Models (LVLMs) have demonstrated impressive performance on multimodal tasks through scaled architectures and extensive training. Recent studies introduce Mixt…
AutoMCU: Feasibility-First MCU Neural Network Customization via LLM-based Multi-Agent Systems
Penglin Dai, Zijie Zhou, Xincao Xu +3
Deploying neural networks on microcontroller units (MCUs) is critical for edge intelligence but remains challenging due to tight memory, storage, and computation constraints. Exist…