Memory in humans and deep language models: Linking hypotheses for model augmentation

Omri Raccah, Phoebe Chen, Ted L. Willke +2

The computational complexity of the self-attention mechanism in Transformer models significantly limits their ability to generalize over long temporal durations. Memory-augmentatio…