Continual Low-Rank Scaled Dot-product Attention

Ginés Carreto Picón, Illia Oleksiienko, Lukas Hedegaard +2

Transformers are widely used for their ability to capture data relations in sequence processing, with great success for a wide range of static tasks. However, the computational and…