Fast Weight Attention for Continual Learning

Yifan Zhang, Steve Ta, Jasper Zhang +8

Recurrent fast-weight memories and selective state-space models compress an expanding context into a fixed-size recurrent state, making the state transition an online learning rule…