AUBER: Automated BERT Regularization

Hyun Dong Lee, Seongmin Lee, U Kang

How can we effectively regularize BERT? Although BERT proves its effectiveness in various downstream natural language processing tasks, it often overfits when there are only a smal…