2.1k citations · 5.6k across the 30 of their papers we have counts for
Showing 2021Show all
3 papers · 1 filter
cs.CV2021★ 201 cited
Masked Autoencoders Are Scalable Vision Learners
Kaiming He, Xinlei Chen, Saining Xie +3
This paper shows that masked autoencoders (MAE) are scalable self-supervised learners for computer vision. Our MAE approach is simple: we mask random patches of the input image and…
cs.CV2021★ 17 cited
An Empirical Study of Training Self-Supervised Vision Transformers
Xinlei Chen, Saining Xie, Kaiming He
This paper does not describe a novel method. Instead, it studies a straightforward, incremental, yet must-know baseline given the recent progress in computer vision: self-supervise…
cs.LG2021★ 35 cited
Understanding self-supervised Learning Dynamics without Contrastive Pairs
Yuandong Tian, Xinlei Chen, Surya Ganguli
While contrastive approaches of self-supervised learning (SSL) learn representations by minimizing the distance between two augmented views of the same data point (positive pairs)…