Knowledge Distillation for BERT Unsupervised Domain Adaptation

Minho Ryu, Kichun Lee

A pre-trained language model, BERT, has brought significant performance improvements across a range of natural language processing tasks. Since the model is trained on a large corp…