Visualizing the loss landscape of Self-supervised Vision Transformer

Youngwan Lee, Jeffrey Ryan Willette, Jonghee Kim +1

The Masked autoencoder (MAE) has drawn attention as a representative self-supervised approach for masked image modeling with vision transformers. However, even though MAE shows bet…