2 papers
cs.DC2025
PolyServe: Efficient Multi-SLO Serving at Scale
Kan Zhu, Haiyang Shi, Le Xu +4
Advances in Large Language Models (LLMs) have led to a surge of LLM-powered applications. These applications have diverse token-generation latency requirements. As a result, simply…
cs.DC2025
AIBrix: Towards Scalable, Cost-Effective Large Language Model Inference Infrastructure
The AIBrix Team, Jiaxin Shan, Varun Gupta +24
We introduce AIBrix, a cloud-native, open-source framework designed to optimize and simplify large-scale LLM deployment in cloud environments. Unlike traditional cloud-native stack…