Understanding the Effect of using Semantically Meaningful Tokens for Visual Representation Learning

Neha Kalibhat, Priyatham Kattakinda, Sumit Nawathe +5

Vision transformers have established a precedent of patchifying images into uniformly-sized chunks before processing. We hypothesize that this design choice may limit models in lea…