Resource-Efficient Pruning for Transformer via Low-Rank Importance Estimation

Peng Liu, Huibing Zeng, Yiqun Zhang +2

With the rapid development of large-scale pre-trained language models based on Transformer architectures, their high computational and memory costs have become a major obstacle to…