28 citations · 36 across the 20 of their papers we have counts for
17 papers · 1 filter
Cloud-native and Distributed Systems for Efficient and Scalable Large Language Models -- A Research Agenda
Minxian Xu, Jingfeng Wu, Shengye Song +16
The rapid rise of Large Language Models (LLMs) has revolutionized various artificial intelligence (AI) applications, from natural language processing to code generation. However, t…
Multi-Layer Scheduling for MoE-Based LLM Reasoning
Yifan Sun, Gholamreza Haffari, Minxian Xu +2
Large Language Models (LLMs) have achieved remarkable success across a wide range of tasks, but serving them efficiently at scale remains a critical challenge due to their substant…
DOPD: A Dynamic PD-Disaggregation Architecture for Maximizing Goodput in LLM Inference Serving
Junhan Liao, Minxian Xu, Wanyi Zheng +4
To meet strict Service-Level Objectives (SLOs),contemporary Large Language Models (LLMs) decouple the prefill and decoding stages and place them on separate GPUs to mitigate the di…
BanaServe: Unified KV Cache and Dynamic Module Migration for Balancing Disaggregated LLM Serving in AI Infrastructure
Yiyuan He, Minxian Xu, Jingfeng Wu +7
Large language models (LLMs) are increasingly deployed in AI infrastructure, driving the need for high throughput, resource efficient serving systems. Disaggregated LLM serving, wh…
TD3-Sched: Learning to Orchestrate Container-based Cloud-Edge Resources via Distributed Reinforcement Learning
Shengye Song, Minxian Xu, Kan Hu +2
Resource scheduling in cloud-edge systems is challenging as edge nodes run latency-sensitive workloads under tight resource constraints, while existing centralized schedulers can s…
Cloud Native System for LLM Inference Serving
Minxian Xu, Junhan Liao, Jingfeng Wu +3
Large Language Models (LLMs) are revolutionizing numerous industries, but their substantial computational demands create challenges for efficient deployment, particularly in cloud…
Unlock the Potential of Fine-grained LLM Serving via Dynamic Module Scaling
Jingfeng Wu, Yiyuan He, Minxian Xu +3
The rise of large language models (LLMs) has created new opportunities across various fields but has also introduced significant challenges in resource management. Current LLM serv…
C-Koordinator: Interference-aware Management for Large-scale and Co-located Microservice Clusters
Shengye Song, Minxian Xu, Zuowei Zhang +5
Microservices transform traditional monolithic applications into lightweight, loosely coupled application components and have been widely adopted in many enterprises. Cloud platfor…
BrownoutServe: SLO-Aware Inference Serving under Bursty Workloads for MoE-based LLMs
Jianmin Hu, Minxian Xu, Kejiang Ye +1
In recent years, the Mixture-of-Experts (MoE) architecture has been widely applied to large language models (LLMs), providing a promising solution that activates only a subset of t…
BucketServe: Bucket-Based Dynamic Batching for Smart and Efficient LLM Inference Serving
Wanyi Zheng, Minxian Xu, Shengye Song +1
Large language models (LLMs) have become increasingly popular in various areas, traditional business gradually shifting from rule-based systems to LLM-based solutions. However, the…
Auto-scaling Approaches for Microservice Applications: A Survey and Taxonomy
Minxian Xu, Junhan Liao, Linfeng Wen +4
Microservice applications are created as loosely coupled application components and they leverage cloud elasticity to reduce costs and increase development speed. However, microser…
SealOS+: A Sealos-based Approach for Adaptive Resource Optimization Under Dynamic Workloads for Securities Trading System
Haojie Jia, Zhenhao Li, Gen Li +2
As securities trading systems transition to a microservices architecture, optimizing system performance presents challenges such as inefficient resource scheduling and high service…
LSRAM: A Lightweight Autoscaling and SLO Resource Allocation Framework for Microservices Based on Gradient Descent
Kan Hu, Minxian Xu, Kejiang Ye +1
Microservices architecture has become the dominant architecture in cloud computing paradigm with its advantages of facilitating development, deployment, modularity and scalability.…
UELLM: A Unified and Efficient Approach for LLM Inference Serving
Yiyuan He, Minxian Xu, Jingfeng Wu +3
In the context of Machine Learning as a Service (MLaaS) clouds, the extensive use of Large Language Models (LLMs) often requires efficient management of significant query loads. Wh…
MSARS: A Meta-Learning and Reinforcement Learning Framework for SLO Resource Allocation and Adaptive Scaling for Microservices
Kan Hu, Linfeng Wen, Minxian Xu +1
Service Level Objectives (SLOs) aim to set threshold for service time in cloud services to ensure acceptable quality of service (QoS) and user satisfaction. Currently, many studies…
CloudNativeSim: a toolkit for modeling and simulation of cloud-native applications
Jingfeng Wu, Minxian Xu, Yiyuan He +2
Cloud-native applications are increasingly becoming popular in modern software design. Employing a microservice-based architecture into these applications is a prevalent strategy t…
Enhanced Load Balancing Approach to Avoid Deadlocks in Cloud
K. S. Rashmi, V. Suma, M. Vaidehi
The state-of-art of the technology focuses on data processing to deal with massive amount of data. Cloud computing is an emerging technology, which enables one to accomplish the af…