Avoiding Catastrophe: Active Dendrites Enable Multi-Task Learning in Dynamic Environments

Abstract

A key challenge for AI is to build embodied systems that operate in dynamically changing environments. Such systems must adapt to changing task contexts and learn continuously. Although standard deep learning systems achieve state of the art results on static benchmarks, they often struggle in dynamic scenarios. In these settings, error signals from multiple contexts can interfere with one another, ultimately leading to a phenomenon known as catastrophic forgetting. In this article we investigate biologically inspired architectures as solutions to these problems. Specifically, we show that the biophysical properties of dendrites and local inhibitory systems enable networks to dynamically restrict and route information in a context-specific manner. Our key contributions are as follows. First, we propose a novel artificial neural network architecture that incorporates active dendrites and sparse representations into the standard deep learning framework. Next, we study the performance of this architecture on two separate benchmarks requiring task-based adaptation: Meta-World, a multi-task reinforcement learning environment where a robotic agent must learn to solve a variety of manipulation tasks simultaneously; and a continual learning benchmark in which the model's prediction task changes throughout training. Analysis on both benchmarks demonstrates the emergence of overlapping but distinct and sparse subnetworks, allowing the system to fluidly learn multiple tasks with minimal forgetting. Our neural implementation marks the first time a single architecture has achieved competitive results on both multi-task and continual learning settings. Our research sheds light on how biological properties of neurons can inform deep learning systems to address dynamic scenarios that are typically impossible for traditional ANNs to solve.

arXiv comment31 pages, 17 figures

References in corpus (18)

• Adam: A Method for Stochastic Optimization
• Overcoming catastrophic forgetting in neural networks
• Prototypical Networks for Few-shot Learning
• A Brief Survey of Deep Reinforcement Learning
• Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor
• Continual Learning Through Synaptic Intelligence
• How to Train Your Robot with Deep Reinforcement Learning; Lessons We've Learned
• Three scenarios for continual learning
• GradNorm: Gradient Normalization for Adaptive Loss Balancing in Deep Multitask Networks
• Why Neurons Have Thousands of Synapses, A Theory of Sequence Memory in Neocortex
• Meta-World: A Benchmark and Evaluation for Multi-Task and Meta Reinforcement Learning
• Alleviating catastrophic forgetting using context-dependent gating and synaptic stabilization
• Adapting Auxiliary Losses Using Gradient Similarity
• How Can We Be So Dense? The Benefits of Using Highly Sparse Representations
• How do neurons operate on sparse distributed representations? A mathematical theory of sparsity, neurons and active dendrites
• Reinforcement Learning with Competitive Ensembles of Information-Constrained Primitives
• How to build a cognitive map: insights from models of the hippocampal formation
• Sparsity and Heterogeneous Dropout for Continual Learning in the Null Space of Neural Activations

Cited by in corpus (3)

• Continual Learning with Neuromorphic Computing: Foundations, Methods, and Emerging Applications
• Replay4NCL: An Efficient Memory Replay-based Methodology for Neuromorphic Continual Learning in Embedded AI Systems
• Context selectivity with dynamic availability enables lifelong continual learning