Unraveling Projection Heads in Contrastive Learning: Insights from Expansion and Shrinkage

Yu Gui, Cong Ma, Yiqiao Zhong

We investigate the role of projection heads, also known as projectors, within the encoder-projector framework (e.g., SimCLR) used in contrastive learning. We aim to demystify the o…