Compress then Serve: Serving Thousands of LoRA Adapters with Little Overhead

Rickard Brüel-Gabrielsson, Jiacheng Zhu, Onkar Bhardwaj +4

Fine-tuning large language models (LLMs) with low-rank adaptations (LoRAs) has become common practice, often yielding numerous copies of the same LLM differing only in their LoRA u…