118 citations · 809 across the 58 of their papers we have counts for
Showing 2021 · cs.CLShow all
3 papers · 2 filters
cs.CL2021
What's Hidden in a One-layer Randomly Weighted Transformer?
Sheng Shen, Zhewei Yao, Douwe Kiela +2
We demonstrate that, hidden within one-layer randomly weighted neural networks, there exist subnetworks that can achieve impressive performance, without ever modifying the weight i…
cs.CL2021★ 2 cited
LEAP: Learnable Pruning for Transformer-based Models
Zhewei Yao, Xiaoxia Wu, Linjian Ma +4
Pruning is an effective method to reduce the memory footprint and computational cost associated with large natural language processing models. However, current pruning algorithms e…
cs.CL2021★ 16 cited
I-BERT: Integer-only BERT Quantization
Sehoon Kim, Amir Gholami, Zhewei Yao +2
Transformer based models, like BERT and RoBERTa, have achieved state-of-the-art results in many Natural Language Processing tasks. However, their memory footprint, inference latenc…