Prism: Cost-Efficient Multi-LLM Serving via GPU Memory Ballooning

Shan Yu, Yifan Qiao, Mingyuan Ma +18

Inference providers must maintain availability for many LLMs, including low-volume but essential models, making resource efficiency increasingly important as token prices fall. Ana…