Revisiting Offline Compression: Going Beyond Factorization-based Methods for Transformer Language Models

Mohammadreza Banaei, Klaudia Bałazy, Artur Kasymov +3

Recent transformer language models achieve outstanding results in many natural language processing (NLP) tasks. However, their enormous size often makes them impractical on memory-…