Occam's Razor for Self Supervised Learning: What is Sufficient to Learn Good Representations?

Mark Ibrahim, David Klindt, Randall Balestriero

Deep Learning is often depicted as a trio of data-architecture-loss. Yet, recent Self Supervised Learning (SSL) solutions have introduced numerous additional design choices, e.g.,…