Short-Term Memory in Orthogonal Neural Networks

Olivia L. White, Daniel D. Lee, Haim Sompolinsky

We study the ability of linear recurrent networks obeying discrete time dynamics to store long temporal sequences that are retrievable from the instantaneous state of the network.…