ROSITA: Refined BERT cOmpreSsion with InTegrAted techniques

Yuanxin Liu, Zheng Lin, Fengcheng Yuan

Pre-trained language models of the BERT family have defined the state-of-the-arts in a wide range of NLP tasks. However, the performance of BERT-based models is mainly driven by th…