Vision Transformers provably learn spatial structure

Samy Jelassi, Michael E. Sander, Yuanzhi Li

Vision Transformers (ViTs) have achieved comparable or superior performance than Convolutional Neural Networks (CNNs) in computer vision. This empirical breakthrough is even more r…