Transfer Learning or Self-supervised Learning? A Tale of Two Pretraining Paradigms

Xingyi Yang, Xuehai He, Yuxiao Liang +3

Pretraining has become a standard technique in computer vision and natural language processing, which usually helps to improve performance substantially. Previously, the most domin…