2 papers
cs.LG2025
Soup to go: mitigating forgetting during continual learning with model averaging
Anat Kleiman, Gintare Karolina Dziugaite, Jonathan Frankle +2
In continual learning, where task data arrives in a sequence, fine-tuning on later tasks will often lead to performance degradation on earlier tasks. This is especially pronounced…
cs.LG2024
Scaling Laws for Precision
Tanishq Kumar, Zachary Ankner, Benjamin F. Spector +6
Low precision training and inference affect both the quality and cost of language models, but current scaling laws do not account for this. In this work, we devise "precision-aware…