RefBERT: Compressing BERT by Referencing to Pre-computed Representations

Xinyi Wang, Haiqin Yang, Liang Zhao +2

Recently developed large pre-trained language models, e.g., BERT, have achieved remarkable performance in many downstream natural language processing applications. These pre-traine…