Serving Heterogeneous LoRA Adapters in Distributed LLM Inference Systems

Shashwat Jaiswal, Shrikara Arun, Anjaly Parayil +8

Low-Rank Adaptation (LoRA) has become the de facto method for parameter-efficient fine-tuning of large language models (LLMs), enabling rapid adaptation to diverse domains. In prod…