Centroid-centered Modeling for Efficient Vision Transformer Pre-training

Xin Yan, Zuchao Li, Lefei Zhang

Masked Image Modeling (MIM) is a new self-supervised vision pre-training paradigm using a Vision Transformer (ViT). Previous works can be pixel-based or token-based, using original…