4 citations · 11 across the 17 of their papers we have counts for
1 paper · 1 filter
Dan Busbridge, Amitis Shidani, Floris Weers +3
We propose a distillation scaling law that estimates distilled model performance based on a compute budget and its allocation between the student and teacher. Our findings mitigate…