Architecture-Agnostic Masked Image Modeling -- From ViT back to CNN

Siyuan Li, Di Wu, Fang Wu +2

Masked image modeling, an emerging self-supervised pre-training method, has shown impressive success across numerous downstream vision tasks with Vision transformers. Its underlyin…