Can Vision Transformers Perform Convolution?

Shanda Li, Xiangning Chen, Di He +1

Several recent studies have demonstrated that attention-based networks, such as Vision Transformer (ViT), can outperform Convolutional Neural Networks (CNNs) on several computer vi…