Source-Optimal Training is Transfer-Suboptimal

C. Evans Hedges

We prove that training a source model optimally for its own task is generically suboptimal when the objective is downstream transfer. We study the source-side optimization problem…