2 citations · 2 across the 3 of their papers we have counts for
3 papers
Training-Free Tokenizer Transplantation via Orthogonal Matching Pursuit
Charles Goddard, Fernando Fernandes Neto
We present a training-free method to transplant tokenizers in pretrained large language models (LLMs) by reconstructing unseen token embeddings via Orthogonal Matching Pursuit (OMP…
Domain Adaptation of Llama3-70B-Instruct through Continual Pre-Training and Model Merging: A Comprehensive Evaluation
Shamane Siriwardhana, Mark McQuade, Thomas Gauthier +8
We conducted extensive experiments on domain adaptation of the Meta-Llama-3-70B-Instruct model on SEC data, exploring its performance on both general and domain-specific benchmarks…
Spectrum: Targeted Training on Signal to Noise Ratio
Eric Hartford, Lucas Atkins, Fernando Fernandes Neto +1
Efficiently post-training large language models remains a challenging task due to the vast computational resources required. We present Spectrum, a method that accelerates LLM trai…