Posterior Meta-Replay for Continual Learning
arXiv:2103.01133
Abstract
Learning a sequence of tasks without access to i.i.d. observations is a widely studied form of continual learning (CL) that remains challenging. In principle, Bayesian learning directly applies to this setting, since recursive and one-off Bayesian updates yield the same result. In practice, however, recursive updating often leads to poor trade-off solutions across tasks because approximate inference is necessary for most models of interest. Here, we describe an alternative Bayesian approach where task-conditioned parameter distributions are continually inferred from data. We offer a practical deep learning implementation of our framework based on probabilistic task-conditioned hypernetworks, an approach we term posterior meta-replay. Experiments on standard benchmarks show that our probabilistic hypernetworks compress sequences of posterior parameter distributions with virtually no forgetting. We obtain considerable performance gains compared to existing Bayesian CL methods, and identify task inference as our major limiting factor. This limitation has several causes that are independent of the considered sequential setting, opening up new avenues for progress in CL.
arXiv commentPublished at NeurIPS 2021
References in corpus (39)
- Auto-Encoding Variational Bayes
- Distilling the Knowledge in a Neural Network
- Overcoming catastrophic forgetting in neural networks
- Continual Learning Through Synaptic Intelligence
- Can You Trust Your Model's Uncertainty? Evaluating Predictive Uncertainty Under Dataset Shift
- An Empirical Investigation of Catastrophic Forgetting in Gradient-Based Neural Networks
- Three scenarios for continual learning
- Variational Dropout and the Local Reparameterization Trick
- New insights and perspectives on the natural gradient method
- Overcoming Catastrophic Forgetting by Incremental Moment Matching
- Multiplicative Normalizing Flows for Variational Bayesian Neural Networks
- Generative replay with feedback connections as a general strategy for continual learning
- Online Structured Laplace Approximations For Overcoming Catastrophic Forgetting
- Towards Robust Evaluations of Continual Learning
- Understanding Measures of Uncertainty for Adversarial Example Detection
- Adversarial Variational Bayes: Unifying Variational Autoencoders and Generative Adversarial Networks
- Revisiting Natural Gradient for Deep Networks
- Decomposition of Uncertainty in Bayesian Deep Learning for Efficient and Risk-sensitive Learning
- On Quadratic Penalties in Elastic Weight Consolidation
- A Neural Dirichlet Process Mixture Model for Task-Free Continual Learning
- Functional Regularisation for Continual Learning with Gaussian Processes
- Coresets via Bilevel Optimization for Continual Learning and Streaming
- Variational Inference using Implicit Distributions
- Task Agnostic Continual Learning Using Online Variational Bayes
- Implicit Weight Uncertainty in Neural Networks
- Task Agnostic Continual Learning via Meta Learning
- Continual Deep Learning by Functional Regularisation of Memorable Past
- Gradient Estimators for Implicit Models
- A Unifying Bayesian View of Continual Learning
- A Spectral Approach to Gradient Estimation for Implicit Distributions
- Radial Bayesian Neural Networks: Beyond Discrete Support In Large-Scale Bayesian Deep Learning
- Cramer-Rao Lower Bound and Information Geometry
- Principled Weight Initialization for Hypernetworks
- Improving and Understanding Variational Continual Learning
- Task Agnostic Continual Learning Using Online Variational Bayes with Fixed-Point Updates
- Stein Neural Sampler
- Bayesian Hypernetworks
- Probabilistic Meta-Representations Of Neural Networks
- Hypermodels for Exploration