WavSpA: Wavelet Space Attention for Boosting Transformers' Long Sequence Learning Ability

Yufan Zhuang, Zihan Wang, Fangbo Tao +1

Transformer and its variants are fundamental neural architectures in deep learning. Recent works show that learning attention in the Fourier space can improve the long sequence lea…