Axial Attention in Multidimensional Transformers

Jonathan Ho, Nal Kalchbrenner, Dirk Weissenborn +1

We propose Axial Transformers, a self-attention-based autoregressive model for images and other data organized as high dimensional tensors. Existing autoregressive models either su…