Keep It SimPool: Who Said Supervised Transformers Suffer from Attention Deficit?

Bill Psomas, Ioannis Kakogeorgiou, Konstantinos Karantzalos +1

Convolutional networks and vision transformers have different forms of pairwise interactions, pooling across layers and pooling at the end of the network. Does the latter really ne…