Echo State Transformer: Attention Over Finite Memories

Yannis Bendi-Ouis, Xavier Hinaut

While Large Language Models and their underlying Transformer architecture are remarkably efficient, they do not reflect how our brain processes and learns a diversity of cognitive…