Multiple Choice Learning of Low-Rank Adapters for Language Modeling

Victor Letzelter, Hugo Malard, Mathieu Fontaine +4

We propose LoRA-MCL, a training scheme that extends next-token prediction in language models with a method designed to decode diverse, plausible sentence continuations at inference…