517 citations · 842 across the 38 of their papers we have counts for
Showing cs.CVShow all
2 papers · 1 filter
cs.CV2022★ 7 cited
Exploring The Role of Mean Teachers in Self-supervised Masked Auto-Encoders
Youngwan Lee, Jeffrey Willette, Jonghee Kim +2
Masked image modeling (MIM) has become a popular strategy for self-supervised learning~(SSL) of visual representations with Vision Transformers. A representative MIM model, the mas…
cs.CV2022★ 4 cited
Self-Distillation for Further Pre-training of Transformers
Seanie Lee, Minki Kang, Juho Lee +2
Pre-training a large transformer model on a massive amount of unlabeled data and fine-tuning it on labeled datasets for diverse downstream tasks has proven to be a successful strat…