3 papers
cs.LG2025
Reflecting on the State of Rehearsal-free Continual Learning with Pretrained Models
Lukas Thede, Karsten Roth, Olivier J. Hénaff +2
With the advent and recent ubiquity of foundation models, continual learning (CL) has recently shifted from continual training from scratch to the continual adaptation of pretraine…
cs.CL2025
Investigating Continual Pretraining in Large Language Models: Insights and Implications
ÃaÄatay Yıldız, Nishaanth Kanna Ravichandran, Nitin Sharma +2
Continual learning (CL) in large language models (LLMs) is an evolving domain that focuses on developing efficient and sustainable training strategies to adapt models to emerging k…
cs.CL2024
Adaptation Odyssey in LLMs: Why Does Additional Pretraining Sometimes Fail to Improve?
Fırat Ãncel, Matthias Bethge, Beyza Ermis +3
In the last decade, the generalization and adaptation abilities of deep learning models were typically evaluated on fixed training and test distributions. Contrary to traditional d…