Token-Level Adaptation of LoRA Adapters for Downstream Task Generalization

Joshua Belofsky

This paper introduces a method for adapting LoRA adapters in smaller-sized language models to arbitrary downstream tasks. Unlike standard mixture-of-expert architectures, our metho…