DIET-CP: Lightweight and Data Efficient Self Supervised Continued Pretraining

Bryan Rodas, Natalie Montesino, Jakob Ambsdorf +2

Continued pretraining offers a promising solution for adapting foundation models to a new target domain. However, in specialized domains, available datasets are often very small, l…