Association: Remind Your GAN not to Forget

Abstract

Neural networks are susceptible to catastrophic forgetting. They fail to preserve previously acquired knowledge when adapting to new tasks. Inspired by human associative memory system, we propose a brain-like approach that imitates the associative learning process to achieve continual learning. We design a heuristics mechanism to potentiatively stimulate the model, which guides the model to recall the historical episodes based on the current circumstance and obtained association experience. Besides, a distillation measure is added to depressively alter the efficacy of synaptic transmission, which dampens the feature reconstruction learning for new task. The framework is mediated by potentiation and depression stimulation that play opposing roles in directing synaptic and behavioral plasticity. It requires no access to the original data and is more similar to human cognitive process. Experiments demonstrate the effectiveness of our method in alleviating catastrophic forgetting on image-to-image translation tasks.

References in corpus (14)

• Distilling the Knowledge in a Neural Network
• Overcoming catastrophic forgetting in neural networks
• Continual Learning Through Synaptic Intelligence
• Perceptual Adversarial Networks for Image-to-Image Transformation
• DualGAN: Unsupervised Dual Learning for Image-to-Image Translation
• Overcoming Catastrophic Forgetting by Incremental Moment Matching
• Lifelong Learning with Dynamically Expandable Networks
• Continual Learning in Generative Adversarial Nets
• Generative Face Completion
• RankSRGAN: Generative Adversarial Networks with Ranker for Image Super-Resolution
• Generative Adversarial Network Training is a Continual Learning Problem
• Localizing Catastrophic Forgetting in Neural Networks
• Towards Lifelong Self-Supervision For Unpaired Image-to-Image Translation
• Generative and Discriminative Learning for Distorted Image Restoration