Orchestrating the Development Lifecycle of Machine Learning-Based IoT Applications: A Taxonomy and Survey
arXiv:1910.05433
Abstract
Machine Learning (ML) and Internet of Things (IoT) are complementary advances: ML techniques unlock complete potentials of IoT with intelligence, and IoT applications increasingly feed data collected by sensors into ML models, thereby employing results to improve their business processes and services. Hence, orchestrating ML pipelines that encompasses model training and implication involved in holistic development lifecycle of an IoT application often leads to complex system integration. This paper provides a comprehensive and systematic survey on the development lifecycle of ML-based IoT application. We outline core roadmap and taxonomy, and subsequently assess and compare existing standard techniques used in individual stage.
arXiv comment50 pages, Accepted by ACM Computing Surveys (CSUR)
References in corpus (85)
- Adam: A Method for Stochastic Optimization
- Distilling the Knowledge in a Neural Network
- Continuous control with deep reinforcement learning
- MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications
- Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation
- Communication-Efficient Learning of Deep Networks from Decentralized Data
- Deep learning for time series classification: a review
- Trust Region Policy Optimization
- The Effectiveness of Data Augmentation in Image Classification using Deep Learning
- Accurate, Large Minibatch SGD: Training ImageNet in 1 Hour
- Addressing Function Approximation Error in Actor-Critic Methods
- Ensemble Adversarial Training: Attacks and Defenses
- A Tutorial on Principal Component Analysis
- Deep Speech 2: End-to-End Speech Recognition in English and Mandarin
- Binarized Neural Networks: Training Deep Neural Networks with Weights and Activations Constrained to +1 or -1
- FitNets: Hints for Thin Deep Nets
- Prioritized Experience Replay
- Soft Actor-Critic Algorithms and Applications
- Feature Squeezing: Detecting Adversarial Examples in Deep Neural Networks
- Dueling Network Architectures for Deep Reinforcement Learning
- DoReFa-Net: Training Low Bitwidth Convolutional Neural Networks with Low Bitwidth Gradients
- Federated Optimization: Distributed Machine Learning for On-Device Intelligence
- Quantized Neural Networks: Training Neural Networks with Low Precision Weights and Activations
- HOGWILD!: A Lock-Free Approach to Parallelizing Stochastic Gradient Descent
- Targeted Backdoor Attacks on Deep Learning Systems Using Data Poisoning
- cuDNN: Efficient Primitives for Deep Learning
- Deep Gradient Compression: Reducing the Communication Bandwidth for Distributed Training
- One weird trick for parallelizing convolutional neural networks
- BadNets: Identifying Vulnerabilities in the Machine Learning Model Supply Chain
- Towards Federated Learning at Scale: System Design
- QSGD: Communication-Efficient SGD via Gradient Quantization and Encoding
- A Survey of Model Compression and Acceleration for Deep Neural Networks
- Learning Differentially Private Recurrent Language Models
- Poisoning Attacks against Support Vector Machines
- Emergence of Locomotion Behaviours in Rich Environments
- TernGrad: Ternary Gradients to Reduce Communication in Distributed Deep Learning
- Efficient Neural Architecture Search via Parameter Sharing
- Revisiting Distributed Synchronous SGD
- Don't Decay the Learning Rate, Increase the Batch Size
- Group Equivariant Convolutional Networks
- Optimal Distributed Online Prediction using Mini-Batches
- Ensembles of Deep LSTM Learners for Activity Recognition using Wearables
- Horovod: fast and easy distributed deep learning in TensorFlow
- Large Batch Training of Convolutional Networks
- Learning Structured Sparsity in Deep Neural Networks
- Deep Reinforcement Learning for Dialogue Generation
- SMASH: One-Shot Model Architecture Search through HyperNetworks
- Understanding Adversarial Training: Increasing Local Stability of Neural Nets through Robust Optimization
- Speeding-up Convolutional Neural Networks Using Fine-tuned CP-Decomposition
- Training and Inference with Integers in Deep Neural Networks
- Fast Training of Convolutional Networks through FFTs
- Defense-GAN: Protecting Classifiers Against Adversarial Attacks Using Generative Models
- Generating Adversarial Examples with Adversarial Networks
- Device Placement Optimization with Reinforcement Learning
- Parallel Coordinate Descent for L1-Regularized Loss Minimization
- Don't Use Large Mini-Batches, Use Local SGD
- Detection of Unauthorized IoT Devices Using Machine Learning Techniques
- Bayesian Dark Knowledge
- Measuring Catastrophic Forgetting in Neural Networks
- Meta-Learning: A Survey
- A Berkeley View of Systems Challenges for AI
- Building high-level features using large scale unsupervised learning
- A Linearly-Convergent Stochastic L-BFGS Algorithm
- MONAS: Multi-Objective Neural Architecture Search using Reinforcement Learning
- Online Learning: A Comprehensive Survey
- Supporting Very Large Models using Automatic Dataflow Graph Partitioning
- Beyond Data and Model Parallelism for Deep Neural Networks
- Parallel training of DNNs with Natural Gradient and Parameter Averaging
- Taming the Wild: A Unified Analysis of Hogwild!-Style Algorithms
- NSML: Meet the MLaaS platform with a real-world case study
- PipeDream: Fast and Efficient Pipeline Parallel DNN Training
- DCN+: Mixed Objective and Deep Residual Coattention for Question Answering
- Parallel SGD: When does averaging help?
- Optimizing Performance of Recurrent Neural Networks on GPUs
- Efficient and Robust Parallel DNN Training through Model Parallelism on Multi-GPU Platform
- Decoupled Parallel Backpropagation with Convergence Guarantee
- Learning in a Large Function Space: Privacy-Preserving Mechanisms for SVM Learning
- XPipe: Efficient Pipeline Model Parallelism for Multi-GPU DNN Training
- When Neurons Fail
- CYCLADES: Conflict-free Asynchronous Machine Learning
- AMPNet: Asynchronous Model-Parallel Training for Dynamic Neural Networks
- Large-Scale Distributed Second-Order Optimization Using Kronecker-Factored Approximate Curvature for Deep Convolutional Neural Networks
- Distributed Hessian-Free Optimization for Deep Neural Network
- Theoretical Limits of Pipeline Parallel Optimization and Application to Distributed Deep Learning
- Adversarial Reinforcement Learning under Partial Observability in Autonomous Computer Network Defence