9 papers
Can Agentic Trading Systems Pay for Their Own Intelligence?
Qiqi Duan, Changlun Li, Chen Wang +10
Large language model (LLM) agents are increasingly used in trading systems, where model reasoning, tool use, and continual decisions incur costs that are expected to produce tradin…
Benchmarking LLM Serving Systems for Agentic AI Workloads with XPerf
Michael Wang, Yikang Yue, Shaobo Li +3
We present XPerf, a benchmarking framework that load-tests LLM serving systems with diverse agentic AI workloads. It provides detailed profiling of the serving system and hardware,…
SLIM: Saturation-Aware Lightweight Performance Modeling for LLM Serving
Pol G. Recasens, Ferran Agullo, Yue Zhu +3
Large language model (LLM) serving commonly increases batch size to improve throughput, but performance eventually reaches a deployment-dependent plateau beyond which larger batche…
Full-Pipeline Inference Optimization for MiMo-V2.5 Series: Pushing Hybrid SWA Efficiency to the Limit
Xiaomi MiMo Team, Anqi Liu, Aoxin Ma +28
We present a full-pipeline inference optimization for the MiMo-V2.5 model family, which combines Hybrid Sliding Window Attention (Hybrid SWA), sparse Mixture-of-Experts (MoE), and…
TuneAhead: Predicting Fine-tuning Performance Before Full Training Begins
Yuxiang Luo, Haonan Long, Chen Wang +6
Fine-tuning large language models (LLMs) is compute-intensive and error-prone: model performance depends sensitively on data quality and hyperparameter choices, and naïve runs can…
A Risk Decomposition Framework for Pre-Hoc Fine-Tuning Prediction
Yuxiang Luo, Chen Wang, Nan Tang
The high cost of fine-tuning LLMs poses a significant economic barrier; pre-hoc performance prediction offers a critical solution to substantially reduce this expense. However, the…
Towards Efficient Key-Value Cache Management for Prefix Prefilling in LLM Inference
Yue Zhu, Hao Yu, Chen Wang +2
The increasing adoption of large language models (LLMs) with extended context windows necessitates efficient Key-Value Cache (KVC) management to optimize inference performance. Inf…
Mind the Memory Gap: Unveiling GPU Bottlenecks in Large-Batch LLM Inference
Pol G. Recasens, Ferran Agullo, Yue Zhu +5
Large language models have been widely adopted across different tasks, but their auto-regressive generation nature often leads to inefficient resource utilization during inference.…
A Robust Power Model Training Framework for Cloud Native Runtime Energy Metric Exporter
Sunyanan Choochotkaew, Chen Wang, Huamin Chen +4
Estimating power consumption in modern Cloud environments is essential for carbon quantification toward green computing. Specifically, it is important to properly account for the p…