Differentially Private Continual Learning
arXiv:1902.06497
Abstract
Catastrophic forgetting can be a significant problem for institutions that must delete historic data for privacy reasons. For example, hospitals might not be able to retain patient data permanently. But neural networks trained on recent data alone will tend to forget lessons learned on old data. We present a differentially private continual learning framework based on variational inference. We estimate the likelihood of past data given the current model using differentially private generative models of old datasets.
Presented at the Privacy in Machine Learning and AI workshop at ICML 2018
References in corpus (7)
- Overcoming catastrophic forgetting in neural networks
- Deep Learning with Differential Privacy
- DeCAF: A Deep Convolutional Activation Feature for Generic Visual Recognition
- How transferable are features in deep neural networks?
- Continual Learning Through Synaptic Intelligence
- Semi-supervised Knowledge Transfer for Deep Learning from Private Training Data
- Less-forgetting Learning in Deep Neural Networks