Coral: Cost-Efficient Multi-LLM Serving over Heterogeneous Cloud GPUs

Yixuan Mei, Zikun Li, Zixuan Chen +5

The usage of large language models (LLMs) has grown increasingly fragmented, with no single model dominating. Meanwhile, cloud providers offer a wide range of mid-tier and older-ge…