When Softmax Fails at the Top: Extreme Value Corrections for InfoNCE

Melihcan Erol, Suat Evren, Oktay Ozel +3

InfoNCE is the standard contrastive learning objective, but its softmax form is not only a computational convenience: it also encodes a statistical assumption about how the top-sco…