Predicting Mergeability of Parameter-Efficient Fine-Tuning Updates

Lin Tang, Wei Zhang, Jing Li +3

Low-rank adaptation (LoRA) makes it cheap to train many domain- and task-specific language model adapters, but whether two adapters can be merged is usually discovered only after b…