Reawakening knowledge: Anticipatory recovery from catastrophic interference via structured training

Yanlai Yang, Matt Jones, Michael C. Mozer +1

We explore the training dynamics of neural networks in a structured non-IID setting where documents are presented cyclically in a fixed, repeated sequence. Typically, networks suff…