Enhancing Performance of Vision Transformers on Small Datasets through Local Inductive Bias Incorporation

Ibrahim Batuhan Akkaya, Senthilkumar S. Kathiresan, Elahe Arani +1

Vision transformers (ViTs) achieve remarkable performance on large datasets, but tend to perform worse than convolutional neural networks (CNNs) when trained from scratch on smalle…