Improving Autoregressive NLP Tasks via Modular Linearized Attention

Victor Agostinelli, Lizhong Chen

Various natural language processing (NLP) tasks necessitate models that are efficient and small based on their ultimate application at the edge or in other resource-constrained env…