Neurogenesis Deep Learning

Abstract

Neural machine learning methods, such as deep neural networks (DNN), have achieved remarkable success in a number of complex data processing tasks. These methods have arguably had their strongest impact on tasks such as image and audio processing - data processing domains in which humans have long held clear advantages over conventional algorithms. In contrast to biological neural systems, which are capable of learning continuously, deep artificial networks have a limited ability for incorporating new information in an already trained network. As a result, methods for continuous learning are potentially highly impactful in enabling the application of deep networks to dynamic data sets. Here, inspired by the process of adult neurogenesis in the hippocampus, we explore the potential for adding new neurons to deep layers of artificial neural networks in order to facilitate their acquisition of novel information while preserving previously trained data representations. Our results on the MNIST handwritten digit dataset and the NIST SD 19 dataset, which includes lower and upper case letters and digits, demonstrate that neurogenesis is well suited for addressing the stability-plasticity dilemma that has long challenged adaptive machine learning algorithms.

8 pages, 8 figures, Accepted to 2017 International Joint Conference on Neural Networks (IJCNN 2017)

References in corpus (3)

• Deep Learning in Neural Networks: An Overview
• How transferable are features in deep neural networks?
• Generative Class-conditional Autoencoders

Cited by in corpus (21)

• FearNet: Brain-Inspired Model for Incremental Learning
• Correction of AI systems by linear discriminants: Probabilistic foundations
• Orthogonal Gradient Descent for Continual Learning
• Incremental Deep Neural Network Learning using Classification Confidence Thresholding
• Replay in Deep Learning: Current Approaches and Missing Biological Elements
• Continuous learning of spiking networks trained with local rules
• The Next Big Thing(s) in Unsupervised Machine Learning: Five Lessons from Infant Learning
• Continual Deep Reinforcement Learning with Task-Agnostic Policy Distillation
• A Self-Commissioning Edge Computing Method for Data-Driven Anomaly Detection in Power Electronic Systems
• NEUZZ: Efficient Fuzzing with Neural Program Smoothing
• Thalamocortical contribution to solving credit assignment in neural systems
• Reviewing continual learning from the perspective of human-level intelligence
• Learn-Prune-Share for Lifelong Learning
• Continual Learning: Tackling Catastrophic Forgetting in Deep Neural Networks with Replay Processes
• On Catastrophic Interference in Atari 2600 Games
• Accretionary Learning with Deep Neural Networks
• Dynamic Nested Hierarchies: Pioneering Self-Evolution in Machine Learning Architectures for Lifelong Intelligence
• FFNB: Forgetting-Free Neural Blocks for Deep Continual Visual Learning
• Towards Heterogeneous Clients with Elastic Federated Learning
• Continuous Learning for Large-scale Personalized Domain Classification
• Incremental Class Learning using Variational Autoencoders with Similarity Learning