Representation Alignment from a Few Paired Data via Distilled Optimal Transport

Julien Kordes, Reuben Dorent

Aligning two independently pretrained unimodal encoders typically demands millions of paired examples. In this paper, we study the regime where only $N\sim 100$--$500$ pairs are a…