66 citations · 147 across the 33 of their papers we have counts for
36 papers · 1 filter
Thinking at the Right Size: Amortized Distillation Across Post-Trained LLMs
Yan Zhou, Sara Kangaslahti, Jonathan Geuter +4
Practical deployment of large language models (LLMs) requires families of post-trained variants---instruction-tuned, reasoning-tuned, and chat-style models---each at multiple sizes…
Domain-Aware Scaling Laws Uncover Data Synergy
Kimia Hamidieh, Lester Mackey, David Alvarez-Melis
Machine learning progress is often attributed to scaling model size and dataset volume, yet the composition of data can be just as consequential. Empirical findings repeatedly show…
Understanding Layer Patching in Model Size Interpolation
Sara Kangaslahti, Jonathan Geuter, Nihal V. Nayak +3
Zero-shot model size interpolation aims to create new models of intermediate target sizes by combining existing models without additional training. Recent work on boomerang distill…
OT on the Map: Quantifying Domain Shifts in Geographic Space
Haoran Zhang, Livia Betti, Konstantin Klemmer +2
In computer vision and machine learning for geographic data, out-of-domain generalization is a pervasive challenge, arising from uneven global data coverage and distribution shifts…
A Critical Look at Targeted Instruction Selection: Disentangling What Matters (and What Doesn't)
Nihal V. Nayak, Paula Rodriguez-Diaz, Neha Hulkund +2
Instruction fine-tuning of large language models (LLMs) often involves selecting a subset of instruction training data from a large candidate pool, using a small query set from the…
Stop Training for the Worst: Progressive Unmasking Accelerates Masked Diffusion Training
Jaeyeon Kim, Jonathan Geuter, David Alvarez-Melis +2
Masked Diffusion Models (MDMs) have emerged as a promising approach for generative modeling in discrete spaces. By generating sequences in any order and allowing for parallel decod…