70 citations · 167 across the 14 of their papers we have counts for
Showing 2021Show all
2 papers · 1 filter
cs.CL2021★ 8 cited
Sparse Progressive Distillation: Resolving Overfitting under Pretrain-and-Finetune Paradigm
Shaoyi Huang, Dongkuan Xu, Ian E. H. Yen +8
Conventional wisdom in pruning Transformer-based language models is that pruning reduces the model expressiveness and thus is more likely to underfit rather than overfit. However,…
cs.CL2021★ 6 cited
Rethinking Network Pruning -- under the Pre-train and Fine-tune Paradigm
Dongkuan Xu, Ian E. H. Yen, Jinxi Zhao +1
Transformer-based pre-trained language models have significantly improved the performance of various natural language processing (NLP) tasks in the recent years. While effective an…