Enabling On-Device Self-Supervised Contrastive Learning With Selective Data Contrast
arXiv:2106.03796
Abstract
After a model is deployed on edge devices, it is desirable for these devices to learn from unlabeled data to continuously improve accuracy. Contrastive learning has demonstrated its great potential in learning from unlabeled data. However, the online input data are usually none independent and identically distributed (non-iid) and storages of edge devices are usually too limited to store enough representative data from different data classes. We propose a framework to automatically select the most representative data from the unlabeled input stream, which only requires a small data buffer for dynamic learning. Experiments show that accuracy and learning speed are greatly improved.
References in corpus (7)
- A Simple Framework for Contrastive Learning of Visual Representations
- Towards Federated Learning at Scale: System Design
- Coresets via Bilevel Optimization for Continual Learning and Streaming
- Self-supervised learning through the eyes of a child
- Accelerating Deep Learning by Focusing on the Biggest Losers
- A Survey of Deep Learning Techniques for Mobile Robot Applications
- Intermittent Inference with Nonuniformly Compressed Multi-Exit Neural Network for Energy Harvesting Powered Devices