NAS-BERT: Task-Agnostic and Adaptive-Size BERT Compression with Neural Architecture Search

Jin Xu, Xu Tan, Renqian Luo +4

While pre-trained language models (e.g., BERT) have achieved impressive results on different natural language processing tasks, they have large numbers of parameters and suffer fro…