Linear Attention with Global Context: A Multipole Attention Mechanism for Vision and Physics

Alex Colagrande, Paul Caillon, Eva Feillet +1

Transformers have become the de facto standard for a wide range of tasks, from image classification to physics simulations. Despite their impressive performance, the quadratic comp…