CAPT: Contrastive Pre-Training for Learning Denoised Sequence Representations

Fuli Luo, Pengcheng Yang, Shicheng Li +2

Pre-trained self-supervised models such as BERT have achieved striking success in learning sequence representations, especially for natural language processing. These models typica…