Tangram: Accelerating Serverless LLM Loading through GPU Memory Reuse and Affinity

Wenbin Zhu, Zhaoyan Shen, Zili Shao +2

Serverless Large Language Models (LLMs) have emerged as a cost-effective solution for deploying AI services by enabling a 'pay-as-you-go' pricing model through GPU resource sharing…