3 citations · 3 across the 8 of their papers we have counts for
Showing 2023Show all
3 papers · 1 filter
cs.IR2023
On Elastic Language Models
Chen Zhang, Benyou Wang, Dawei Song
Large-scale pretrained language models have achieved compelling performance in a wide range of language understanding and information retrieval tasks. Knowledge distillation offers…
cs.CL2023
Sparse Contrastive Learning of Sentence Embeddings
Ruize An, Chen Zhang, Dawei Song
Recently, SimCSE has shown the feasibility of contrastive learning in training sentence embeddings and illustrates its expressiveness in spanning an aligned and uniform embedding s…
cs.CL2023★ 3 cited
Towards the Law of Capacity Gap in Distilling Language Models
Chen Zhang, Qiuchi Li, Dawei Song +3
Language model (LM) distillation aims at distilling the knowledge in a large teacher LM to a small student one. As a critical issue facing LM distillation, a superior student often…