Fast Adaptive Task Offloading in Edge Computing based on Meta Reinforcement Learning

Abstract

Multi-access edge computing (MEC) aims to extend cloud service to the network edge to reduce network traffic and service latency. A fundamental problem in MEC is how to efficiently offload heterogeneous tasks of mobile applications from user equipment (UE) to MEC hosts. Recently, many deep reinforcement learning (DRL) based methods have been proposed to learn offloading policies through interacting with the MEC environment that consists of UE, wireless channels, and MEC hosts. However, these methods have weak adaptability to new environments because they have low sample efficiency and need full retraining to learn updated policies for new environments. To overcome this weakness, we propose a task offloading method based on meta reinforcement learning, which can adapt fast to new environments with a small number of gradient updates and samples. We model mobile applications as Directed Acyclic Graphs (DAGs) and the offloading policy by a custom sequence-to-sequence (seq2seq) neural network. To efficiently train the seq2seq network, we propose a method that synergizes the first order approximation and clipped surrogate objective. The experimental results demonstrate that this new offloading method can reduce the latency by up to 25% compared to three baselines while being able to adapt fast to new environments.

Accepted by IEEE Transaction on Parallel and Distributed Systems

References in corpus (10)

• Adam: A Method for Stochastic Optimization
• Neural Machine Translation by Jointly Learning to Align and Translate
• Continuous control with deep reinforcement learning
• Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks
• High-Dimensional Continuous Control Using Generalized Advantage Estimation
• Deep Reinforcement Learning for Online Computation Offloading in Wireless Powered Mobile-Edge Computing Networks
• On First-Order Meta-Learning Algorithms
• Efficient Off-Policy Meta-Reinforcement Learning via Probabilistic Context Variables
• Online Placement of Multi-Component Applications in Edge Computing Environments
• Meta-Learning: A Survey

Cited by in corpus (8)

• Deep Reinforcement Learning-based Methods for Resource Scheduling in Cloud Computing: A Review and Future Directions
• Multi-objective application placement in fog computing using graph neural network-based reinforcement learning
• Digital Twin-assisted Reinforcement Learning for Resource-aware Microservice Offloading in Edge Computing
• A Survey on Intelligent Computation Offloading and Pricing Strategy in UAV-Enabled MEC Network: Challenges and Research Directions
• A Generic Service-Oriented Function Offloading Framework for Connected Automated Vehicles
• FastAdaBelief: Improving Convergence Rate for Belief-based Adaptive Optimizers by Exploiting Strong Convexity
• A Deep Learning Based Resource Allocator for Communication Networks with Dynamic User Utility Demands
• A Distributed Deep Reinforcement Learning Technique for Application Placement in Edge and Fog Computing Environments