LRC-BERT: Latent-representation Contrastive Knowledge Distillation for Natural Language Understanding

Hao Fu, Shaojun Zhou, Qihong Yang +4

The pre-training models such as BERT have achieved great results in various natural language processing problems. However, a large number of parameters need significant amounts of…