Outlier Suppression: Pushing the Limit of Low-bit Transformer Language Models

Xiuying Wei, Yunchen Zhang, Xiangguo Zhang +5

Transformer architecture has become the fundamental element of the widespread natural language processing~(NLP) models. With the trends of large NLP models, the increasing memory a…