eLLM: Elastic Memory Management Framework for Efficient LLM Serving

Jiale Xu, Rui Zhang, Yi Xiong +11

Large Language Models are increasingly being deployed in datacenters. Serving these models requires careful memory management, as their memory usage includes static weights, dynami…