NOLA: Compressing LoRA using Linear Combination of Random Basis

Soroush Abbasi Koohpayegani, KL Navaneet, Parsa Nooralinejad +2

Fine-tuning Large Language Models (LLMs) and storing them for each downstream task or domain is impractical because of the massive model size (e.g., 350GB in GPT-3). Current litera…