Decoding the Functional Roles of Register and High-Norm Patch Tokens in Vision Transformers

Neel Varma, Andrew Rufail, Dipika Khullar +1

Self-supervised Vision Transformers (ViTs), such as DINOv2, learn rich visual representations, but the functions of their internal tokens remain poorly understood. Recent architect…