RSAC: Regularized Subspace Approximation Classifier for Lightweight Continuous Learning
arXiv:2007.01480
Abstract
Continuous learning seeks to perform the learning on the data that arrives from time to time. While prior works have demonstrated several possible solutions, these approaches require excessive training time as well as memory usage. This is impractical for applications where time and storage are constrained, such as edge computing. In this work, a novel training algorithm, regularized subspace approximation classifier (RSAC), is proposed to achieve lightweight continuous learning. RSAC contains a feature reduction module and classifier module with regularization. Extensive experiments show that RSAC is more efficient than prior continuous learning works and outperforms these works on various experimental settings.
References in corpus (18)
- Deep Learning in Neural Networks: An Overview
- Distilling the Knowledge in a Neural Network
- Overcoming catastrophic forgetting in neural networks
- Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms
- DeCAF: A Deep Convolutional Activation Feature for Generic Visual Recognition
- A Survey on Explainable Artificial Intelligence (XAI): Towards Medical XAI
- A continual learning survey: Defying forgetting in classification tasks
- PathNet: Evolution Channels Gradient Descent in Super Neural Networks
- Rich feature hierarchies for accurate object detection and semantic segmentation
- FearNet: Brain-Inspired Model for Incremental Learning
- Progress & Compress: A scalable framework for continual learning
- Alleviating catastrophic forgetting using context-dependent gating and synaptic stabilization
- RANet: Ranking Attention Network for Fast Video Object Segmentation
- Continual Learning with Deep Generative Replay
- Generative replay with feedback connections as a general strategy for continual learning
- Deep Generative Dual Memory Network for Continual Learning
- AdaNet: Adaptive Structural Learning of Artificial Neural Networks
- End-to-End Incremental Learning