4 papers
LatentBox: Storing AI-Generated Images at Scale via a Latent-First Design
Zirui Wang, Yunjia Zheng, Tingfeng Lan +4
The explosive growth of AI-generated images has created a sustainability challenge for storage infrastructure. Platforms like Midjourney and Adobe Firefly already host billions of…
C2CServe: Leveraging NVLink-C2C for Elastic Serverless LLM Serving on MIG
Shutian Luo, Ali Zafar Sadiq, Rui Yang +4
Modern LLM serving is increasingly serverless in shape: large model catalogs, long-tail invocations, and multi-tenant demand. Existing GPU serving systems face a tradeoff: dedicate…
λScale: Enabling Fast Scaling for Serverless Large Language Model Inference
Minchen Yu, Rui Yang, Chaobo Jia +9
Serverless computing has emerged as a compelling solution for cloud-based model inference. However, as modern large language models (LLMs) continue to grow in size, existing server…
ZenFlow: Enabling Stall-Free Offloading Training via Asynchronous Updates
Tingfeng Lan, Yusen Wu, Bin Ma +7
Fine-tuning large language models (LLMs) often exceeds GPU memory limits, prompting systems to offload model states to CPU memory. However, existing offloaded training frameworks l…