Contrastive Loss is All You Need to Recover Analogies as Parallel Lines

Narutatsu Ri, Fei-Tzin Lee, Nakul Verma

While static word embedding models are known to represent linguistic analogies as parallel lines in high-dimensional space, the underlying mechanism as to why they result in such g…