DistilCSE: Effective Knowledge Distillation For Contrastive Sentence Embeddings

Chaochen Gao, Xing Wu, Peng Wang +4

Large-scale contrastive learning models can learn very informative sentence embeddings, but are hard to serve online due to the huge model size. Therefore, they often play the role…