Momentum^2 Teacher: Momentum Teacher with Momentum Statistics for Self-Supervised Learning

Zeming Li, Songtao Liu, Jian Sun

In this paper, we present a novel approach, Momentum${}^2$ Teacher, for student-teacher based self-supervised learning. The approach performs momentum update on both network weights…