On the Integration of Self-Attention and Convolution

Xuran Pan, Chunjiang Ge, Rui Lu +4

Convolution and self-attention are two powerful techniques for representation learning, and they are usually considered as two peer approaches that are distinct from each other. In…