Deep Reinforcement Learning for Autonomous Internet of Things: Model, Applications and Challenges
arXiv:1907.09059
Abstract
The Internet of Things (IoT) extends the Internet connectivity into billions of IoT devices around the world, where the IoT devices collect and share information to reflect status of the physical world. The Autonomous Control System (ACS), on the other hand, performs control functions on the physical systems without external intervention over an extended period of time. The integration of IoT and ACS results in a new concept - autonomous IoT (AIoT). The sensors collect information on the system status, based on which the intelligent agents in the IoT devices as well as the Edge/Fog/Cloud servers make control decisions for the actuators to react. In order to achieve autonomy, a promising method is for the intelligent agents to leverage the techniques in the field of artificial intelligence, especially reinforcement learning (RL) and deep reinforcement learning (DRL) for decision making. In this paper, we first provide a tutorial of DRL, and then propose a general model for the applications of RL/DRL in AIoT. Next, a comprehensive survey of the state-of-art research on DRL for AIoT is presented, where the existing works are classified and summarized under the umbrella of the proposed general DRL model. Finally, the challenges and open issues for future research are identified.
References in corpus (29)
- Continuous control with deep reinforcement learning
- A Brief Survey of Deep Reinforcement Learning
- Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor
- Trust Region Policy Optimization
- Addressing Function Approximation Error in Actor-Critic Methods
- Prioritized Experience Replay
- Dueling Network Architectures for Deep Reinforcement Learning
- Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environments
- QT-Opt: Scalable Deep Reinforcement Learning for Vision-Based Robotic Manipulation
- Scalable trust-region method for deep reinforcement learning using Kronecker-factored approximation
- Cellular-Connected UAVs over 5G: Deep Reinforcement Learning for Interference Management
- Counterfactual Multi-Agent Policy Gradients
- Towards Massive Machine Type Communications in Ultra-Dense Cellular IoT Networks: Current Issues and Machine Learning-Assisted Solutions
- Stabilising Experience Replay for Deep Multi-Agent Reinforcement Learning
- Distributed Distributional Deterministic Policy Gradients
- Lifelong Federated Reinforcement Learning: A Learning Architecture for Navigation in Cloud Robotic Systems
- Decentralized Computation Offloading for Multi-User Mobile Edge Computing: A Deep Reinforcement Learning Approach
- Memory-based control with recurrent neural networks
- Sample Efficient Actor-Critic with Experience Replay
- Formulation of Deep Reinforcement Learning Architecture Toward Autonomous Driving for On-Ramp Merge
- Unsupervised Predictive Memory in a Goal-Directed Agent
- Playing FPS Games with Deep Reinforcement Learning
- Q-Prop: Sample-Efficient Policy Gradient with An Off-Policy Critic
- Learning to Communicate to Solve Riddles with Deep Distributed Recurrent Q-Networks
- On Improving Deep Reinforcement Learning for POMDPs
- Machine Learning for Vehicular Networks
- Reinforcement Learning-based Energy Trading for Microgrids
- Vehicular Edge Computing via Deep Reinforcement Learning
- Reinforcement Learning-based Resource Allocation in Fog RAN for IoT with Heterogeneous Latency Requirements