4 citations · 13 across the 20 of their papers we have counts for
8 papers · 2 filters
DOPD: A Dynamic PD-Disaggregation Architecture for Maximizing Goodput in LLM Inference Serving
Junhan Liao, Minxian Xu, Wanyi Zheng +4
To meet strict Service-Level Objectives (SLOs),contemporary Large Language Models (LLMs) decouple the prefill and decoding stages and place them on separate GPUs to mitigate the di…
BanaServe: Unified KV Cache and Dynamic Module Migration for Balancing Disaggregated LLM Serving in AI Infrastructure
Yiyuan He, Minxian Xu, Jingfeng Wu +7
Large language models (LLMs) are increasingly deployed in AI infrastructure, driving the need for high throughput, resource efficient serving systems. Disaggregated LLM serving, wh…
FlexPipe: Adapting Dynamic LLM Serving Through Inflight Pipeline Refactoring in Fragmented Serverless Clusters
Yanying Lin, Shijie Peng, Chengzhi Lu +2
Serving Large Language Models (LLMs) in production faces significant challenges from highly variable request patterns and severe resource fragmentation in serverless clusters. Curr…
Cloud Native System for LLM Inference Serving
Minxian Xu, Junhan Liao, Jingfeng Wu +3
Large Language Models (LLMs) are revolutionizing numerous industries, but their substantial computational demands create challenges for efficient deployment, particularly in cloud…
Unlock the Potential of Fine-grained LLM Serving via Dynamic Module Scaling
Jingfeng Wu, Yiyuan He, Minxian Xu +3
The rise of large language models (LLMs) has created new opportunities across various fields but has also introduced significant challenges in resource management. Current LLM serv…
C-Koordinator: Interference-aware Management for Large-scale and Co-located Microservice Clusters
Shengye Song, Minxian Xu, Zuowei Zhang +5
Microservices transform traditional monolithic applications into lightweight, loosely coupled application components and have been widely adopted in many enterprises. Cloud platfor…
BrownoutServe: SLO-Aware Inference Serving under Bursty Workloads for MoE-based LLMs
Jianmin Hu, Minxian Xu, Kejiang Ye +1
In recent years, the Mixture-of-Experts (MoE) architecture has been widely applied to large language models (LLMs), providing a promising solution that activates only a subset of t…
Auto-scaling Approaches for Microservice Applications: A Survey and Taxonomy
Minxian Xu, Junhan Liao, Linfeng Wen +4
Microservice applications are created as loosely coupled application components and they leverage cloud elasticity to reduce costs and increase development speed. However, microser…