Provable Benefits of Unsupervised Pre-training and Transfer Learning via Single-Index Models

Taj Jones-McCormick, Aukosh Jagannath, Subhabrata Sen

Unsupervised pre-training and transfer learning are commonly used techniques to initialize training algorithms for neural networks, particularly in settings with limited labeled da…