Where Should I Spend My FLOPS? Efficiency Evaluations of Visual Pre-training Methods

Skanda Koppula, Yazhe Li, Evan Shelhamer +5

Self-supervised methods have achieved remarkable success in transfer learning, often achieving the same or better accuracy than supervised pre-training. Most prior work has done so…