C2CServe: Leveraging NVLink-C2C for Elastic Serverless LLM Serving on MIG

Shutian Luo, Ali Zafar Sadiq, Rui Yang +4

Modern LLM serving is increasingly serverless in shape: large model catalogs, long-tail invocations, and multi-tenant demand. Existing GPU serving systems face a tradeoff: dedicate…