4 papers · 1 filter
Benchmarking LLM Serving Systems for Agentic AI Workloads with XPerf
Michael Wang, Yikang Yue, Shaobo Li +3
We present XPerf, a benchmarking framework that load-tests LLM serving systems with diverse agentic AI workloads. It provides detailed profiling of the serving system and hardware,…
SLIM: Saturation-Aware Lightweight Performance Modeling for LLM Serving
Pol G. Recasens, Ferran Agullo, Yue Zhu +3
Large language model (LLM) serving commonly increases batch size to improve throughput, but performance eventually reaches a deployment-dependent plateau beyond which larger batche…
Mind the Memory Gap: Unveiling GPU Bottlenecks in Large-Batch LLM Inference
Pol G. Recasens, Ferran Agullo, Yue Zhu +5
Large language models have been widely adopted across different tasks, but their auto-regressive generation nature often leads to inefficient resource utilization during inference.…
A Robust Power Model Training Framework for Cloud Native Runtime Energy Metric Exporter
Sunyanan Choochotkaew, Chen Wang, Huamin Chen +4
Estimating power consumption in modern Cloud environments is essential for carbon quantification toward green computing. Specifically, it is important to properly account for the p…