QUEST: A robust attention formulation using query-modulated spherical attention

Hariprasath Govindarajan, Per Sidén, Jacob Roll +1

The Transformer model architecture has become one of the most widely used in deep learning and the attention mechanism is at its core. The standard attention formulation uses a sof…