Explaining How Deep Neural Networks Forget by Deep Visualization

Abstract

Explaining the behaviors of deep neural networks, usually considered as black boxes, is critical especially when they are now being adopted over diverse aspects of human life. Taking the advantages of interpretable machine learning (interpretable ML), this paper proposes a novel tool called Catastrophic Forgetting Dissector (or CFD) to explain catastrophic forgetting in continual learning settings. We also introduce a new method called Critical Freezing based on the observations of our tool. Experiments on ResNet articulate how catastrophic forgetting happens, particularly showing which components of this famous network are forgetting. Our new continual learning algorithm defeats various recent techniques by a significant margin, proving the capability of the investigation. Critical freezing not only attacks catastrophic forgetting but also exposes explainability.

12 pages, 4 figures, 1 table. arXiv admin note: substantial text overlap with arXiv:2001.01578. Keywords: XAI, Catastrophic Forgetting, Attribution map, Continual Learning and Regularization

References in corpus (9)

• Overcoming catastrophic forgetting in neural networks
• Understanding Neural Networks Through Deep Visualization
• Continual Learning Through Synaptic Intelligence
• Visualizing Deep Neural Network Decisions: Prediction Difference Analysis
• Measuring Catastrophic Forgetting in Neural Networks
• Incremental Learning for Semantic Segmentation of Large-Scale Remote Sensing Data
• Toward Understanding Catastrophic Forgetting in Continual Learning
• ContCap: A scalable framework for continual image captioning
• A Conceptual Framework for Lifelong Learning