Sliceformer: Make Multi-head Attention as Simple as Sorting in Discriminative Tasks

Shen Yuan, Hongteng Xu

As one of the most popular neural network modules, Transformer plays a central role in many fundamental deep learning models, e.g., the ViT in computer vision and the BERT and GPT…