Efficient Masked Autoencoders with Self-Consistency

Zhaowen Li, Yousong Zhu, Zhiyang Chen +5

Inspired by the masked language modeling (MLM) in natural language processing tasks, the masked image modeling (MIM) has been recognized as a strong self-supervised pre-training me…