Score the Update, Not the Token: Descent-Aligned Routing for Combinatorial LoRA Experts

Priya Nair, Lukas Brenner, Maya Lindqvist +5

Mixture-of-LoRA-experts methods raise the capacity of low-rank adaptation by routing each token to a few low-rank experts. Nearly all of them tie one input-side factor to one outpu…