Incorporating Neuro-Inspired Adaptability for Continual Learning in Artificial Intelligence

Abstract

Continual learning aims to empower artificial intelligence (AI) with strong adaptability to the real world. For this purpose, a desirable solution should properly balance memory stability with learning plasticity, and acquire sufficient compatibility to capture the observed distributions. Existing advances mainly focus on preserving memory stability to overcome catastrophic forgetting, but remain difficult to flexibly accommodate incremental changes as biological intelligence (BI) does. By modeling a robust Drosophila learning system that actively regulates forgetting with multiple learning modules, here we propose a generic approach that appropriately attenuates old memories in parameter distributions to improve learning plasticity, and accordingly coordinates a multi-learner architecture to ensure solution compatibility. Through extensive theoretical and empirical validation, our approach not only clearly enhances the performance of continual learning, especially over synaptic regularization methods in task-incremental settings, but also potentially advances the understanding of neurological adaptive mechanisms, serving as a novel paradigm to progress AI and BI together.

References in corpus (9)

• Overcoming catastrophic forgetting in neural networks
• A continual learning survey: Defying forgetting in classification tasks
• Riemannian Walk for Incremental Learning: Understanding Forgetting and Intransigence
• PathNet: Evolution Channels Gradient Descent in Super Neural Networks
• Progress & Compress: A scalable framework for continual learning
• Learning to Learn without Forgetting by Maximizing Transfer and Minimizing Interference
• Memory Replay with Data Compression for Continual Learning
• AFEC: Active Forgetting of Negative Transfer in Continual Learning
• AI of Brain and Cognitive Sciences: From the Perspective of First Principles

Cited by in corpus (8)

• Continual Learning and Catastrophic Forgetting
• MAGR: Manifold-Aligned Graph Regularization for Continual Action Quality Assessment
• Continual Learning with Neuromorphic Computing: Foundations, Methods, and Emerging Applications
• Quantum continual learning on a programmable superconducting processor
• Towards Incremental Learning in Large Language Models: A Critical Review
• CL-BioGAN: Biologically-Inspired Cross-Domain Continual Learning for Hyperspectral Anomaly Detection
• Continual Learning in the Presence of Repetition
• Weighted Ensemble Models Are Strong Continual Learners