Does Mapping Non-Maximal Probabilities to GMM Components Matter for S-JEPA Encoder Representations?

Wenxuan He, Yunpeng Li, Shan Liang

S-JEPA uses soft Gaussian mixture model (GMM) posteriors instead of hard cluster labels to preserve uncertainty. It remains unclear whether the probability values alone are suffici…