Pervasive AI for IoT applications: A Survey on Resource-efficient Distributed Artificial Intelligence

Abstract

Artificial intelligence (AI) has witnessed a substantial breakthrough in a variety of Internet of Things (IoT) applications and services, spanning from recommendation systems to robotics control and military surveillance. This is driven by the easier access to sensory data and the enormous scale of pervasive/ubiquitous devices that generate zettabytes (ZB) of real-time data streams. Designing accurate models using such data streams, to predict future insights and revolutionize the decision-taking process, inaugurates pervasive systems as a worthy paradigm for a better quality-of-life. The confluence of pervasive computing and artificial intelligence, Pervasive AI, expanded the role of ubiquitous IoT systems from mainly data collection to executing distributed computations with a promising alternative to centralized learning, presenting various challenges. In this context, a wise cooperation and resource scheduling should be envisaged among IoT devices (e.g., smartphones, smart vehicles) and infrastructure (e.g. edge nodes, and base stations) to avoid communication and computation overheads and ensure maximum performance. In this paper, we conduct a comprehensive survey of the recent techniques developed to overcome these resource challenges in pervasive AI systems. Specifically, we first present an overview of the pervasive computing, its architecture, and its intersection with artificial intelligence. We then review the background, applications and performance metrics of AI, particularly Deep Learning (DL) and online learning, running in a ubiquitous system. Next, we provide a deep literature review of communication-efficient techniques, from both algorithmic and system perspectives, of distributed inference, training and online learning tasks across the combination of IoT devices, edge devices and cloud servers. Finally, we discuss our future vision and research challenges.

arXiv commentSurvey paper accepted in IEEE Communications Surveys & Tutorials

References in corpus (66)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Deep Learning with Differential Privacy
• Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms
• Federated Learning: Challenges, Methods, and Future Directions
• Generative Adversarial Networks
• A Brief Survey of Deep Reinforcement Learning
• End to End Learning for Self-Driving Cars
• A Survey of the Recent Architectures of Deep Convolutional Neural Networks
• A Survey on Explainable Artificial Intelligence (XAI): Towards Medical XAI
• MXNet: A Flexible and Efficient Machine Learning Library for Heterogeneous Distributed Systems
• Federated Learning with Non-IID Data
• Client Selection for Federated Learning with Heterogeneous Resources in Mobile Edge
• Convergence of Edge Computing and Deep Learning: A Comprehensive Survey
• Federated Learning for Internet of Things: A Comprehensive Survey
• Deep Reinforcement Learning for Multi-Agent Systems: A Review of Challenges, Solutions and Applications
• Edge Intelligence: The Confluence of Edge Computing and Artificial Intelligence
• Deep Learning with Limited Numerical Precision
• Learning to Communicate with Deep Multi-Agent Reinforcement Learning
• Speech Commands: A Dataset for Limited-Vocabulary Speech Recognition
• On the Convergence of FedAvg on Non-IID Data
• A Survey on Distributed Machine Learning
• Classification with Noisy Labels by Importance Reweighting
• A Survey and Critique of Multiagent Deep Reinforcement Learning
• Cost-Effective Active Learning for Deep Image Classification
• Sparse Communication for Distributed Gradient Descent
• Federated Learning for 6G Communications: Challenges, Methods, and Future Directions
• Fully Decentralized Multi-Agent Reinforcement Learning with Networked Agents
• A Crowdsourcing Framework for On-Device Federated Learning
• Learning Attentional Communication for Multi-Agent Cooperation
• Actor-Attention-Critic for Multi-Agent Reinforcement Learning
• Edge Computing For Smart Health: Context-aware Approaches, Opportunities, and Challenges
• Optimal and Approximate Q-value Functions for Decentralized POMDPs
• Federated Learning in the Sky: Aerial-Ground Air Quality Sensing Framework with UAV Swarms
• Deep Models Under the GAN: Information Leakage from Collaborative Deep Learning
• Differential Privacy-enabled Federated Learning for Sensitive Health Data
• I Know What You See: Power Side-Channel Attack on Convolutional Neural Network Accelerators
• CINIC-10 is not ImageNet or CIFAR-10
• JALAD: Joint Accuracy- and Latency-Aware Deep Structure Decoupling for Edge-Cloud Execution
• A Hybrid Deep Learning Architecture for Privacy-Preserving Mobile Analytics
• Active Self-Paced Learning for Cost-Effective and Progressive Face Identification
• Analysis and Optimal Edge Assignment For Hierarchical Federated Learning on Non-IID Data
• Enabling AI in Future Wireless Networks: A Data Life Cycle Perspective
• Accelerating Federated Learning over Reliability-Agnostic Clients in Mobile Edge Computing Systems
• Distributed Training of Deep Learning Models: A Taxonomic Perspective
• Optimization for Reinforcement Learning: From Single Agent to Cooperative Agents
• Deep Learning Based Video System for Accurate and Real-Time Parking Measurement
• FLaaS: Federated Learning as a Service
• Decentralized Cooperative Stochastic Bandits
• R-MADDPG for Partially Observable Environments and Limited Communication
• Distributed CNN Inference on Resource-Constrained UAVs for Surveillance Systems: Design and Optimization
• Ultra-Reliable Low-Latency Vehicular Networks: Taming the Age of Information Tail
• Edge Intelligence: On-Demand Deep Learning Model Co-Inference with Device-Edge Synergy
• A simple local 3-approximation algorithm for vertex cover
• Differentially-Private Federated Linear Bandits
• A Game-Theoretic Approach to Design Secure and Resilient Distributed Support Vector Machines
• Learning Efficient Multi-agent Communication: An Information Bottleneck Approach
• Musical Chair: Efficient Real-Time Recognition Using Collaborative IoT Devices
• Guardians of the Deep Fog: Failure-Resilient DNN Inference from Edge to Cloud
• Federated Bandit: A Gossiping Approach
• RL-DistPrivacy: Privacy-Aware Distributed Deep Inference for low latency IoT systems
• Multi-Agent Multi-Armed Bandits with Limited Communication
• Federated Multi-armed Bandits with Personalization
• Distributed Non-Stochastic Experts
• Combining Federated and Active Learning for Communication-efficient Distributed Failure Prediction in Aeronautics
• Federated Self-supervised Learning for Heterogeneous Clients
• Security of Distributed Machine Learning: A Game-Theoretic Approach to Design Secure DSVM

Cited by in corpus (10)

• Empowering Edge Intelligence: A Comprehensive Survey on On-Device AI Models
• Towards Ubiquitous Semantic Metaverse: Challenges, Approaches, and Opportunities
• A Tutorial on 5G Positioning
• Deep Reinforcement Learning for Trajectory Path Planning and Distributed Inference in Resource-Constrained UAV Swarms
• Optimal Resource Management for Hierarchical Federated Learning over HetNets with Wireless Energy Transfer
• Zero-touch realization of Pervasive Artificial Intelligence-as-a-service in 6G networks
• RL-DistPrivacy: Privacy-Aware Distributed Deep Inference for low latency IoT systems
• A Blockchain-based Reliable Federated Meta-learning for Metaverse: A Dual Game Framework
• Fundamentals of Physical AI
• A Predictive Neural Network Architecture for Early Detection of Low-Rate Cyberattacks