Extremely Low Bit Transformer Quantization for On-Device Neural Machine Translation

Insoo Chung, Byeongwook Kim, Yoonjung Choi +5

The deployment of widely used Transformer architecture is challenging because of heavy computation load and memory overhead during inference, especially when the target device is l…