Trade-offs in Ensembling, Merging and Routing Among Parameter-Efficient Experts

Sanae Lotfi, Lucas Caccia, Alessandro Sordoni +2

While large language models (LLMs) fine-tuned with lightweight adapters achieve strong performance across diverse tasks, their performance on individual tasks depends on the fine-t…