Small Models Scout Bottleneck Order for Large-Model Data Control

Seungmin Choi, Jiwon Sung, Muhammad Umer +4

Small proxy models are commonly used to identify data mixtures for larger-scale training. We ask whether their training trajectories reveal another transferable structure: the orde…