2 papers
cs.LG2025
Soup to go: mitigating forgetting during continual learning with model averaging
Anat Kleiman, Gintare Karolina Dziugaite, Jonathan Frankle +2
In continual learning, where task data arrives in a sequence, fine-tuning on later tasks will often lead to performance degradation on earlier tasks. This is especially pronounced…
cs.LG2024
Transcendence: Generative Models Can Outperform The Experts That Train Them
Edwin Zhang, Vincent Zhu, Naomi Saphra +5
Generative models are trained with the simple objective of imitating the conditional probability distribution induced by the data they are trained on. Therefore, when trained on da…