Resource Management in Wireless Networks via Multi-Agent Deep Reinforcement Learning

Abstract

We propose a mechanism for distributed resource management and interference mitigation in wireless networks using multi-agent deep reinforcement learning (RL). We equip each transmitter in the network with a deep RL agent that receives delayed observations from its associated users, while also exchanging observations with its neighboring agents, and decides on which user to serve and what transmit power to use at each scheduling interval. Our proposed framework enables agents to make decisions simultaneously and in a distributed manner, unaware of the concurrent decisions of other agents. Moreover, our design of the agents' observation and action spaces is scalable, in the sense that an agent trained on a scenario with a specific number of transmitters and users can be applied to scenarios with different numbers of transmitters and/or users. Simulation results demonstrate the superiority of our proposed approach compared to decentralized baselines in terms of the tradeoff between average and $5^{th}$ percentile user rates, while achieving performance close to, and even in certain cases outperforming, that of a centralized information-theoretic baseline. We also show that our trained agents are robust and maintain their performance gains when experiencing mismatches between train and test deployments.

arXiv commentFinal version to appear in IEEE Transactions on Wireless Communications

References in corpus (19)

• Continuous control with deep reinforcement learning
• Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor
• Dota 2 with Large Scale Deep Reinforcement Learning
• Sim-to-Real Transfer of Robotic Control with Dynamics Randomization
• Multi-Agent Deep Reinforcement Learning for Dynamic Power Allocation in Wireless Networks
• Counterfactual Multi-Agent Policy Gradients
• Learning to reinforcement learn
• Intelligent Power Control for Spectrum Sharing in Cognitive Radios: A Deep Reinforcement Learning Approach
• Quantifying Generalization in Reinforcement Learning
• Deep Decentralized Multi-task Multi-Agent Reinforcement Learning under Partial Observability
• Load-Aware Modeling and Analysis of Heterogeneous Cellular Networks
• On The Power of Curriculum Learning in Training Deep Networks
• A Joint Learning and Communications Framework for Federated Learning over Wireless Networks
• Move Evaluation in Go Using Deep Convolutional Neural Networks
• Learning to Play Guess Who? and Inventing a Grounded Language as a Consequence
• A Deep Q-Learning Method for Downlink Power Allocation in Multi-Cell Networks
• The Reactor: A fast and sample-efficient Actor-Critic agent for Reinforcement Learning
• ZPD Teaching Strategies for Deep Reinforcement Learning from Demonstrations
• Deep Reinforcement Learning for Distributed Uncoordinated Cognitive Radios Resource Allocation

Cited by in corpus (13)

• Learning Decentralized Wireless Resource Allocations with Graph Neural Networks
• A Deep Reinforcement Learning Framework for Contention-Based Spectrum Sharing
• Learning Resilient Radio Resource Management Policies with Graph Neural Networks
• Deep Multi-agent Reinforcement Learning for Highway On-Ramp Merging in Mixed Traffic
• Joint link scheduling and power allocation in imperfect and energy-constrained underwater wireless sensor networks
• State-Augmented Learnable Algorithms for Resource Management in Wireless Networks
• Applications of Multi-Agent Reinforcement Learning in Future Internet: A Comprehensive Survey
• Wireless MAC Protocol Synthesis and Optimization with Multi-Agent Distributed Reinforcement Learning
• Optimizing Resource Allocation for QoS and Stability in Dynamic VLC-NOMA Networks via MARL
• Conservative and Risk-Aware Offline Multi-Agent Reinforcement Learning
• Percentile Optimization in Wireless Networks- Part I: Power Control for Max-Min-Rate to Sum-Rate Maximization (and Everything in Between)
• Wireless Link Scheduling via Graph Representation Learning: A Comparative Study of Different Supervision Levels
• Percentile Optimization in Wireless Networks- Part II: Beamforming for Cell-Edge Throughput Maximization