DeltaZip: Efficient Serving of Multiple Full-Model-Tuned LLMs

Xiaozhe Yao, Qinghao Hu, Ana Klimovic

Fine-tuning large language models (LLMs) greatly improves model quality for downstream tasks. However, serving many fine-tuned LLMs concurrently is challenging due to the sporadic,…