ViT Registers and Fractal ViT

Jason Chuan-Chih Chou, Abhinav Kumar, Shivank Garg

Drawing inspiration from recent findings including surprisingly decent performance of transformers without positional encoding (NoPE) in the domain of language models and how regis…