On-device Training: A First Overview on Existing Systems

Abstract

The recent breakthroughs in machine learning (ML) and deep learning (DL) have catalyzed the design and development of various intelligent systems over wide application domains. While most existing machine learning models require large memory and computing power, efforts have been made to deploy some models on resource-constrained devices as well. A majority of the early application systems focused on exploiting the inference capabilities of ML and DL models, where data captured from different mobile and embedded sensing components are processed through these models for application goals such as classification and segmentation. More recently, the concept of exploiting the mobile and embedded computing resources for ML/DL model training has gained attention, as such capabilities allow (i) the training of models via local data without the need to share data over wireless links, thus enabling privacy-preserving computation by design, (ii) model personalization and environment adaptation, and (ii) deployment of accurate models in remote and hardly accessible locations without stable internet connectivity. This work targets to summarize and analyze state-of-the-art systems research that allows such on-device model training capabilities and provide a survey of on-device training from a systems perspective.

This paper has been accepted to ACM Transactions on Sensor Networks (TOSN), September 2024. DOI: https://doi.org/10.1145/3696003

References in corpus (12)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
• Neural Architecture Search with Reinforcement Learning
• Fast Graph Representation Learning with PyTorch Geometric
• Towards Accurate Binary Convolutional Neural Network
• SPINN: Synergistic Progressive Inference of Neural Networks over Device and Cloud
• ElasticTrainer: Speeding Up On-Device Training with Runtime Elastic Tensor Selection
• Cost-effective On-device Continual Learning over Memory Hierarchy with Miro
• A Graph Theoretic Framework of Recomputation Algorithms for Memory-Efficient Backpropagation
• p-Meta: Towards On-device Deep Model Adaptation
• CGaP: Continuous Growth and Pruning for Efficient Deep Learning
• MNN: A Universal and Efficient Inference Engine

Cited by in corpus (2)

• Efficient Precision-Scalable Hardware for Microscaling (MX) Processing in Robotics Learning
• Precision-Scalable Microscaling Datapaths with Optimized Reduction Tree for Efficient NPU Integration