BPDec: Unveiling the Potential of Masked Language Modeling Decoder in BERT pretraining

Wen Liang, Youzhi Liang

BERT (Bidirectional Encoder Representations from Transformers) has revolutionized the field of natural language processing through its exceptional performance on numerous tasks. Ye…