Understanding Geometric Representations in Self-Supervised Vision Transformers via Subspace Intervention

Weichen Zhou, Yawen Zou, Chunzhi Gu +3

We introduce a controlled subspace intervention framework to investigate how self-supervised Vision Transformers (ViTs) encode dense geometric information. While linear probing is…