Vision Transformers Need Registers

Timothée Darcet, Maxime Oquab, Julien Mairal +1

Transformers have recently emerged as a powerful tool for learning visual representations. In this paper, we identify and characterize artifacts in feature maps of both supervised…