RePre: Improving Self-Supervised Vision Transformer with Reconstructive Pre-training

Luya Wang, Feng Liang, Yangguang Li +3

Recently, self-supervised vision transformers have attracted unprecedented attention for their impressive representation learning ability. However, the dominant method, contrastive…