Confidence Regularized Masked Language Modeling using Text Length

Seunghyun Ji, Soowon Lee

Masked language modeling is a widely used method for learning language representations, where the model predicts a randomly masked word in each input. However, this approach typica…