DINO as a von Mises-Fisher mixture model

Hariprasath Govindarajan, Per Sidén, Jacob Roll +1

Self-distillation methods using Siamese networks are popular for self-supervised pre-training. DINO is one such method based on a cross-entropy loss between $K$-dimensional probabi…