Rethinking Network Pruning -- under the Pre-train and Fine-tune Paradigm

Dongkuan Xu, Ian E. H. Yen, Jinxi Zhao +1

Transformer-based pre-trained language models have significantly improved the performance of various natural language processing (NLP) tasks in the recent years. While effective an…