A Deep Reinforcement Learning Framework for Contention-Based Spectrum Sharing
arXiv:2110.02736 · doi:10.1109/JSAC.2021.3087254
Abstract
The increasing number of wireless devices operating in unlicensed spectrum motivates the development of intelligent adaptive approaches to spectrum access. We consider decentralized contention-based medium access for base stations (BSs) operating on unlicensed shared spectrum, where each BS autonomously decides whether or not to transmit on a given resource. The contention decision attempts to maximize not its own downlink throughput, but rather a network-wide objective. We formulate this problem as a decentralized partially observable Markov decision process with a novel reward structure that provides long term proportional fairness in terms of throughput. We then introduce a two-stage Markov decision process in each time slot that uses information from spectrum sensing and reception quality to make a medium access decision. Finally, we incorporate these features into a distributed reinforcement learning framework for contention-based spectrum access. Our formulation provides decentralized inference, online adaptability and also caters to partial observability of the environment through recurrent Q-learning. Empirically, we find its maximization of the proportional fairness metric to be competitive with a genie-aided adaptive energy detection threshold, while being robust to channel fading and small contention windows.
arXiv comment14 pages, 11 figures, 4 tables
References in corpus (7)
- Adam: A Method for Stochastic Optimization
- Dueling Network Architectures for Deep Reinforcement Learning
- What should 6G be?
- Learning to Communicate with Deep Multi-Agent Reinforcement Learning
- Deep Recurrent Q-Learning for Partially Observable MDPs
- Distributive Dynamic Spectrum Access through Deep Reinforcement Learning: A Reservoir Computing Based Approach
- Resource Management in Wireless Networks via Multi-Agent Deep Reinforcement Learning