Continual Pre-Training Mitigates Forgetting in Language and Vision

Andrea Cossu, Tinne Tuytelaars, Antonio Carta +3

Pre-trained models are nowadays a fundamental component of machine learning research. In continual learning, they are commonly used to initialize the model before training on the s…