3 papers
cs.CL2025
Training-Free Tokenizer Transplantation via Orthogonal Matching Pursuit
Charles Goddard, Fernando Fernandes Neto
We present a training-free method to transplant tokenizers in pretrained large language models (LLMs) by reconstructing unseen token embeddings via Orthogonal Matching Pursuit (OMP…
cs.CL2025
Arcee's MergeKit: A Toolkit for Merging Large Language Models
Charles Goddard, Shamane Siriwardhana, Malikeh Ehghaghi +5
The rapid expansion of the open-source language model landscape presents an opportunity to merge the competencies of these model checkpoints by combining their parameters. Advances…
cs.CL2024
Merging in a Bottle: Differentiable Adaptive Merging (DAM) and the Path from Averaging to Automation
Thomas Gauthier-Caron, Shamane Siriwardhana, Elliot Stein +5
By merging models, AI systems can combine the distinct strengths of separate language models, achieving a balance between multiple capabilities without requiring substantial retrai…