Entropy Reveals Block Importance in Masked Self-Supervised Vision Transformers

Peihao Xiang, Kaida Wu, Ou Bai

Masked self-supervised vision transformers have become a dominant pretraining paradigm, yet their substantial model size poses significant challenges for resource-constrained deplo…