DistrAttention: An Efficient and Flexible Self-Attention Mechanism on Modern GPUs

Haolin Jin, Mengbai Xiao, Yuan Yuan +4

The Transformer architecture has revolutionized deep learning, delivering the state-of-the-art performance in areas such as natural language processing, computer vision, and time s…