ZETA: Leveraging Z-order Curves for Efficient Top-k Attention

Qiuhao Zeng, Jerry Huang, Peng Lu +4

Over recent years, the Transformer has become a fundamental building block for sequence modeling architectures. Yet at its core is the use of self-attention, whose memory and compu…