ABC: Attention with Bounded-memory Control

Hao Peng, Jungo Kasai, Nikolaos Pappas +5

Transformer architectures have achieved state-of-the-art results on a variety of sequence modeling tasks. However, their attention mechanism comes with a quadratic complexity in se…