3 papers
cs.LG2026
Demystifying Numerical Instability in LLM Inference: Achieving Reproducible Inference for Mission-Critical Tasks with HEAL
Zhenting Zhu, Lucas Thai, Shan Yu +5
As Large Language Models (LLMs) deploy into mission-critical domains (e.g., finance, medicine, and law), output reproducibility has become a strict system requirement. While practi…
cs.DC2026
Prism: Cost-Efficient Multi-LLM Serving via GPU Memory Ballooning
Shan Yu, Yifan Qiao, Mingyuan Ma +18
Inference providers must maintain availability for many LLMs, including low-volume but essential models, making resource efficiency increasingly important as token prices fall. Ana…
cs.DC2025
ConServe: Fine-Grained GPU Harvesting for LLM Online and Offline Co-Serving
Yifan Qiao, Shu Anzai, Shan Yu +10
Large language model (LLM) serving demands low latency and high throughput, but high load variability makes it challenging to achieve high GPU utilization. In this paper, we identi…