Good Pretraining, Bad SFT: Checkpoint Quality Across the Training Stack

Sohir Maskey, Philipp Scholl, Jonas Knupp +2

Language-model checkpoints are commonly selected by pretraining loss or benchmark scores, assuming that the highest-scoring checkpoint will remain the best starting point for subse…