Contrastive Learning-based Sentence Encoders Implicitly Weight Informative Words

Hiroto Kurita, Goro Kobayashi, Sho Yokoi +1

The performance of sentence encoders can be significantly improved through the simple practice of fine-tuning using contrastive loss. A natural question arises: what characteristic…