DSFormer: Effective Compression of Text-Transformers by Dense-Sparse Weight Factorization

Rahul Chand, Yashoteja Prabhu, Pratyush Kumar

With the tremendous success of large transformer models in natural language understanding, down-sizing them for cost-effective deployments has become critical. Recent studies have…