ELFATT: Efficient Linear Fast Attention for Vision Transformers

Chong Wu, Maolin Che, Renjie Xu +2

The attention mechanism is the key to the success of transformers in different machine learning tasks. However, the quadratic complexity with respect to the sequence length of the…