Deep Reinforcement Learning for Online Computation Offloading in Wireless Powered Mobile-Edge Computing Networks

Abstract

Wireless powered mobile-edge computing (MEC) has recently emerged as a promising paradigm to enhance the data processing capability of low-power networks, such as wireless sensor networks and internet of things (IoT). In this paper, we consider a wireless powered MEC network that adopts a binary offloading policy, so that each computation task of wireless devices (WDs) is either executed locally or fully offloaded to an MEC server. Our goal is to acquire an online algorithm that optimally adapts task offloading decisions and wireless resource allocations to the time-varying wireless channel conditions. This requires quickly solving hard combinatorial optimization problems within the channel coherence time, which is hardly achievable with conventional numerical optimization methods. To tackle this problem, we propose a Deep Reinforcement learning-based Online Offloading (DROO) framework that implements a deep neural network as a scalable solution that learns the binary offloading decisions from the experience. It eliminates the need of solving combinatorial optimization problems, and thus greatly reduces the computational complexity especially in large-size networks. To further reduce the complexity, we propose an adaptive procedure that automatically adjusts the parameters of the DROO algorithm on the fly. Numerical results show that the proposed algorithm can achieve near-optimal performance while significantly decreasing the computation time by more than an order of magnitude compared with existing optimization methods. For example, the CPU execution latency of DROO is less than $0.1$ second in a $30$-user network, making real-time and optimal offloading truly viable even in a fast fading environment.

arXiv commentThis paper has been accepted for publication by IEEE Transactions on Mobile Computing (Acceptance date: July 2019). If you find the paper interesting, please cite the paper as: L. Huang, S. Bi, and Y. J. Zhang, "Deep reinforcement learning for online computation offloading in wireless powered mobile-edge computing networks," IEEE Trans. Mobile Comput., DOI:10.1109/TMC.2019.2928811, July 2019

References in corpus (3)

• Dynamic Computation Offloading for Mobile-Edge Computing with Energy Harvesting Devices
• Wireless Powered Communication: Opportunities and Challenges
• Deep Reinforcement Learning in Large Discrete Action Spaces

Cited by in corpus (21)

• Edge Intelligence: The Confluence of Edge Computing and Artificial Intelligence
• Fast Adaptive Task Offloading in Edge Computing based on Meta Reinforcement Learning
• Dynamic Scheduling for Stochastic Edge-Cloud Computing Environments using A3C learning and Residual Recurrent Neural Networks
• Delay-aware and Energy-Efficient Computation Offloading in Mobile Edge Computing Using Deep Reinforcement Learning
• Optimal Energy Allocation and Task Offloading Policy for Wireless Powered Mobile Edge Computing Systems
• Enabling AI in Future Wireless Networks: A Data Life Cycle Perspective
• A Review on Computational Intelligence Techniques in Cloud and Edge Computing
• Proximal Policy Optimization-based Transmit Beamforming and Phase-shift Design in an IRS-aided ISAC System for the THz Band
• Reinforcement Learning-Empowered Mobile Edge Computing for 6G Edge Intelligence
• Data Age Aware Scheduling for Wireless Powered Mobile-Edge Computing in Industrial Internet of Things
• DeepEdge: A Deep Reinforcement Learning based Task Orchestrator for Edge Computing
• Deep progressive reinforcement learning-based flexible resource scheduling framework for IRS and UAV-assisted MEC system
• Computation Bits Maximization in a Backscatter Assisted Wirelessly Powered MEC Network
• A Fast Task Offloading Optimization Framework for IRS-Assisted Multi-Access Edge Computing System
• LiteChain: A Lightweight Blockchain for Verifiable and Scalable Federated Learning in Massive Edge Networks
• Optimal Resource Allocation for Multi-user OFDMA-URLLC MEC Systems
• Joint Optimization of Service Caching Placement and Computation Offloading in Mobile Edge Computing Systems
• Spatiotemporal Non-Uniformity-Aware Online Task Scheduling in Collaborative Edge Computing for Industrial Internet of Things
• Digital Twin-assisted Reinforcement Learning for Resource-aware Microservice Offloading in Edge Computing
• A Generic Service-Oriented Function Offloading Framework for Connected Automated Vehicles
• A Deep RL Approach on Task Placement and Scaling of Edge Resources for Cellular Vehicle-to-Network Service Provisioning