CERT: Contrastive Self-supervised Learning for Language Understanding

Hongchao Fang, Sicheng Wang, Meng Zhou +2

Pretrained language models such as BERT, GPT have shown great effectiveness in language understanding. The auxiliary predictive tasks in existing pretraining approaches are mostly…