Compressing Pre-trained Transformers via Low-Bit NxM Sparsity for Natural Language Understanding

Connor Holmes, Minjia Zhang, Yuxiong He +1

In recent years, large pre-trained Transformer networks have demonstrated dramatic improvements in many natural language understanding tasks. However, the huge size of these models…