BCMT: Blockwise Causal Memory Transformer

Rachid Arezki

Transformer architectures rely on dense self-attention to model long-range dependencies, but this mechanism exhibits quadratic complexity with respect to sequence length. We introd…