MC-BERT: Efficient Language Pre-Training via a Meta Controller

Zhenhui Xu, Linyuan Gong, Guolin Ke +5

Pre-trained contextual representations (e.g., BERT) have become the foundation to achieve state-of-the-art results on many NLP tasks. However, large-scale pre-training is computati…