Lite Transformer with Long-Short Range Attention

Zhanghao Wu, Zhijian Liu, Ji Lin +2

Transformer has become ubiquitous in natural language processing (e.g., machine translation, question answering); however, it requires enormous amount of computations to achieve hi…