Mashup Learning: Faster Finetuning by Remixing Past Checkpoints

Sofia Maria Lo Cicero Vaina, Artem Chumachenko, Max Ryabinin

Finetuning on domain-specific data is a well-established method for enhancing LLM performance on downstream tasks. Training on each dataset produces a new set of model weights, res…