Fractional neural attention for efficient multiscale sequence processing

Cheng Kevin Qu, Andrew Ly, Pulin Gong

Attention mechanisms underpin the computational power of Transformer models, which have achieved remarkable success across diverse domains. Yet understanding and extending the prin…