8 citations · 21 across the 10 of their papers we have counts for
Showing cs.CLShow all
2 papers · 1 filter
cs.CL2022
An Automatic and Efficient BERT Pruning for Edge AI Systems
Shaoyi Huang, Ning Liu, Yueying Liang +5
With the yearning for deep learning democratization, there are increasing demands to implement Transformer-based natural language processing (NLP) models on resource-constrained de…
cs.CL2021★ 8 cited
Sparse Progressive Distillation: Resolving Overfitting under Pretrain-and-Finetune Paradigm
Shaoyi Huang, Dongkuan Xu, Ian E. H. Yen +8
Conventional wisdom in pruning Transformer-based language models is that pruning reduces the model expressiveness and thus is more likely to underfit rather than overfit. However,…