Decentralized Computation Offloading for Multi-User Mobile Edge Computing: A Deep Reinforcement Learning Approach

Abstract

Mobile edge computing (MEC) emerges recently as a promising solution to relieve resource-limited mobile devices from computation-intensive tasks, which enables devices to offload workloads to nearby MEC servers and improve the quality of computation experience. Nevertheless, by considering a MEC system consisting of multiple mobile users with stochastic task arrivals and wireless channels in this paper, the design of computation offloading policies is challenging to minimize the long-term average computation cost in terms of power consumption and buffering delay. A deep reinforcement learning (DRL) based decentralized dynamic computation offloading strategy is investigated to build a scalable MEC system with limited feedback. Specifically, a continuous action space-based DRL approach named deep deterministic policy gradient (DDPG) is adopted to learn efficient computation offloading policies independently at each mobile user. Thus, powers of both local execution and task offloading can be adaptively allocated by the learned policies from each user's local observation of the MEC system. Numerical results are illustrated to demonstrate that efficient policies can be learned at each user, and performance of the proposed DDPG based decentralized strategy outperforms the conventional deep Q-network (DQN) based discrete power control strategy and some other greedy strategies with reduced computation cost. Besides, the power-delay tradeoff is also analyzed for both the DDPG based and DQN based strategies.

References in corpus (8)

• Adam: A Method for Stochastic Optimization
• Continuous control with deep reinforcement learning
• A Brief Survey of Deep Reinforcement Learning
• Mobile Edge Computing: A Survey on Architecture and Computation Offloading
• Dynamic Computation Offloading for Mobile-Edge Computing with Energy Harvesting Devices
• Joint Optimization of Radio and Computational Resources for Multicell Mobile-Edge Computing
• Delay-Optimal Computation Task Scheduling for Mobile-Edge Computing Systems
• Learning-Based Computation Offloading for IoT Devices with Energy Harvesting

Cited by in corpus (13)

• Convergence of Edge Computing and Deep Learning: A Comprehensive Survey
• A Review on Computational Intelligence Techniques in Cloud and Edge Computing
• Discontinuous Computation Offloading for Energy-Efficient Mobile Edge Computing
• Joint Optimization of Signal Design and Resource Allocation in Wireless D2D Edge Computing
• Deep Reinforcement Learning for Autonomous Internet of Things: Model, Applications and Challenges
• A Survey on Intelligent Computation Offloading and Pricing Strategy in UAV-Enabled MEC Network: Challenges and Research Directions
• Hierarchical Reinforcement Learning for Relay Selection and Power Optimization in Two-Hop Cooperative Relay Network
• Multi-user Resource Control with Deep Reinforcement Learning in IoT Edge Computing
• Decentralized Multi-Agent Reinforcement Learning for Task Offloading Under Uncertainty
• Distributed Resource Scheduling for Large-Scale MEC Systems: A Multi-Agent Ensemble Deep Reinforcement Learning with Imitation Acceleration
• Deep Reinforcement Learning for Backscatter-Aided Data Offloading in Mobile Edge Computing
• A Deep RL Approach on Task Placement and Scaling of Edge Resources for Cellular Vehicle-to-Network Service Provisioning
• Deep Deterministic Policy Gradient for Relay Selection and Power Allocation in Cooperative Communication Network