Train on Validation (ToV): Fast data selection with applications to fine-tuning

Ayush Jain, Andrea Montanari, Eren Sasoglu

State-of-the-art machine learning often follows a two-stage process: $(i)$~pre-training on large, general-purpose datasets; $(ii)$~fine-tuning on task-specific data. In fine-tuning…