Tensorized Clustered LoRA Merging for Multi-Task Interference

Zhan Su, Fengran Mo, Guojun Liang +4

Despite the success of the monolithic dense paradigm of large language models (LLMs), the LoRA adapters offer an efficient solution by fine-tuning small task-specific modules and m…