Text Compression-aided Transformer Encoding

Zuchao Li, Zhuosheng Zhang, Hai Zhao +4

Text encoding is one of the most important steps in Natural Language Processing (NLP). It has been done well by the self-attention mechanism in the current state-of-the-art Transfo…