6.5k citations · 10.4k across the 118 of their papers we have counts for
4 papers · 2 filters
What's Hidden in a One-layer Randomly Weighted Transformer?
Sheng Shen, Zhewei Yao, Douwe Kiela +2
We demonstrate that, hidden within one-layer randomly weighted neural networks, there exist subnetworks that can achieve impressive performance, without ever modifying the weight i…
Learned Token Pruning for Transformers
Sehoon Kim, Sheng Shen, David Thorsley +4
Deploying transformer models in practice is challenging due to their inference cost, which scales quadratically with input sequence length. To address this, we present a novel Lear…
LEAP: Learnable Pruning for Transformer-based Models
Zhewei Yao, Xiaoxia Wu, Linjian Ma +4
Pruning is an effective method to reduce the memory footprint and computational cost associated with large natural language processing models. However, current pruning algorithms e…
I-BERT: Integer-only BERT Quantization
Sehoon Kim, Amir Gholami, Zhewei Yao +2
Transformer based models, like BERT and RoBERTa, have achieved state-of-the-art results in many Natural Language Processing tasks. However, their memory footprint, inference latenc…