p-Meta: Towards On-device Deep Model Adaptation

Abstract

Data collected by IoT devices are often private and have a large diversity across users. Therefore, learning requires pre-training a model with available representative data samples, deploying the pre-trained model on IoT devices, and adapting the deployed model on the device with local data. Such an on-device adaption for deep learning empowered applications demands data and memory efficiency. However, existing gradient-based meta learning schemes fail to support memory-efficient adaptation. To this end, we propose p-Meta, a new meta learning method that enforces structure-wise partial parameter updates while ensuring fast generalization to unseen tasks. Evaluations on few-shot image classification and reinforcement learning tasks show that p-Meta not only improves the accuracy but also substantially reduces the peak dynamic memory by a factor of 2.5 on average compared to state-of-the-art few-shot adaptation methods.

arXiv commentPublished in SIGKDD 2022

References in corpus (12)

• Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks
• Prototypical Networks for Few-shot Learning
• Trust Region Policy Optimization
• Accurate, Large Minibatch SGD: Training ImageNet in 1 Hour
• Squeeze-and-Excitation Networks
• Benchmarking Deep Reinforcement Learning for Continuous Control
• Meta-Learning for Semi-Supervised Few-Shot Classification
• Training Deep Nets with Sublinear Memory Cost
• The Reversible Residual Network: Backpropagation Without Storing Activations
• Highway and Residual Networks learn Unrolled Iterative Estimation
• BOIL: Towards Representation Change for Few-shot Learning
• Torchmeta: A Meta-Learning library for PyTorch

Cited by in corpus (3)

• On-device Training: A First Overview on Existing Systems
• Hierarchical Training of Deep Neural Networks Using Early Exiting
• AdaShadow: Responsive Test-time Model Adaptation in Non-stationary Mobile Environments