Features are fate: a theory of transfer learning in high-dimensional regression

Javan Tahir, Surya Ganguli, Grant M. Rotskoff

With the emergence of large-scale pre-trained neural networks, methods to adapt such "foundation" models to data-limited downstream tasks have become a necessity. Fine-tuning, pref…