Randomly Initialized Networks Can Learn from Peer-to-Peer Consensus

Esteban Rodríguez-Betancourt, Edgar Casasola-Murillo

In self-supervised learning, self-distilled methods have shown impressive performance, learning representations useful for downstream tasks and even displaying emergent properties.…