DeepCuts: Single-Shot Interpretability based Pruning for BERT

Jasdeep Singh Grover, Bhavesh Gawri, Ruskin Raj Manku

As language models have grown in parameters and layers, it has become much harder to train and infer with them on single GPUs. This is severely restricting the availability of larg…