1 citations · 2 across the 14 of their papers we have counts for
26 papers · 1 filter
CoCoScale: Leveraging Layer-wise Scaling to Unlock the Potential of Online LLM Serving
Jingfeng Wu, Yiyuan He, Minxian Xu +7
Online large language model (LLM) serving has become the backbone of modern AI applications, powering diverse downstream services through shared hardware clusters. However, modern…
BrownoutMoE: Structure-Aware Expert Grouping for Efficient and Accurate LLM Web-based Services
Yi Ding, Minxian Xu, Zhengxin Fang +2
Mixture-of-Experts (MoE) large language models (LLMs) are increasingly deployed in Web-facing services, where inference must be both accurate and responsive under bursty demand. Al…
RolloutPipe: Overlapping Pipelined Rollout and Training in Disaggregated On-Policy LLM Reinforcement Learning
Rongjian Chen, Jianmin Hu, Kejiang Ye +1
Large language model (LLM) post-training for reasoning increasingly relies on reinforcement learning with verifiable rewards (RLVR), where models learn from ground-truth feedback o…
SwiftCache: Efficient LLM Serving for Multi-turn Conversations with Heterogeneous KV Cache Sharing
Jianmin Hu, Minxian Xu, Sa Wang +5
Multi-turn conversation is a fundamental scenario in LLM applications, widely used in chatbots and AI agents. As the conversation evolves, historical tokens accumulate continuously…
Cloud-native and Distributed Systems for Efficient and Scalable Large Language Models -- A Research Agenda
Minxian Xu, Jingfeng Wu, Shengye Song +16
The rapid rise of Large Language Models (LLMs) has revolutionized various artificial intelligence (AI) applications, from natural language processing to code generation. However, t…
DOPD: A Dynamic PD-Disaggregation Architecture for Maximizing Goodput in LLM Inference Serving
Junhan Liao, Minxian Xu, Wanyi Zheng +4
To meet strict Service-Level Objectives (SLOs),contemporary Large Language Models (LLMs) decouple the prefill and decoding stages and place them on separate GPUs to mitigate the di…