CLOOB: Modern Hopfield Networks with InfoLOOB Outperform CLIP

Andreas Fürst, Elisabeth Rumetshofer, Johannes Lehner +8

CLIP yielded impressive results on zero-shot transfer learning tasks and is considered as a foundation model like BERT or GPT3. CLIP vision models that have a rich representation a…