5 papers
A Theory of Contrastive Learning with Natural Images
Antonio Torralba, Yair Weiss
Why does contrastive learning with simple images and augmentations yield useful representations for downstream tasks? We address this question by analytically computing the optimal…
What Color is the Sky (for a non-human) ?
Yair Weiss, Ofer Springer
The light of the daytime sky contains a mixture of many colors yet is perceived as blue by human observers. This is largely due to the particular response functions of the human co…
Is the Modality Gap a Bug or a Feature? A Robustness Perspective
Rhea Chowers, Oshri Naparstek, Udi Barzelay +1
Many modern multi-modal models (e.g. CLIP) seek an embedding space in which the two modalities are aligned. Somewhat surprisingly, almost all existing models show a strong modality…
Characterizing Nonlinear Dynamics via Smooth Prototype Equivalences
Roy Friedman, Noa Moriel, Matthew Ricci +3
Characterizing the long term behavior of dynamical systems given limited measurements is a common challenge throughout the physical and biological sciences. This is a challenging t…
On adversarial training and the 1 Nearest Neighbor classifier
Amir Hagai, Yair Weiss
The ability to fool deep learning classifiers with tiny perturbations of the input has lead to the development of adversarial training in which the loss with respect to adversarial…