Locally Shifted Attention With Early Global Integration

Shelly Sheynin, Sagie Benaim, Adam Polyak +1

Recent work has shown the potential of transformers for computer vision applications. An image is first partitioned into patches, which are then used as input tokens for the attent…