Cellular-Connected UAVs over 5G: Deep Reinforcement Learning for Interference Management

Abstract

In this paper, an interference-aware path planning scheme for a network of cellular-connected unmanned aerial vehicles (UAVs) is proposed. In particular, each UAV aims at achieving a tradeoff between maximizing energy efficiency and minimizing both wireless latency and the interference level caused on the ground network along its path. The problem is cast as a dynamic game among UAVs. To solve this game, a deep reinforcement learning algorithm, based on echo state network (ESN) cells, is proposed. The introduced deep ESN architecture is trained to allow each UAV to map each observation of the network state to an action, with the goal of minimizing a sequence of time-dependent utility functions. Each UAV uses ESN to learn its optimal path, transmission power level, and cell association vector at different locations along its path. The proposed algorithm is shown to reach a subgame perfect Nash equilibrium (SPNE) upon convergence. Moreover, an upper and lower bound for the altitude of the UAVs is derived thus reducing the computational complexity of the proposed algorithm. Simulation results show that the proposed scheme achieves better wireless latency per UAV and rate per ground user (UE) while requiring a number of steps that is comparable to a heuristic baseline that considers moving via the shortest distance towards the corresponding destinations. The results also show that the optimal altitude of the UAVs varies based on the ground network density and the UE data rate requirements and plays a vital role in minimizing the interference level on the ground UEs as well as the wireless transmission delay of the UAV.

References in corpus (3)

• Unmanned Aerial Vehicle with Underlaid Device-to-Device Communications: Performance and Tradeoffs
• The Sky Is Not the Limit: LTE for Unmanned Aerial Vehicles
• Coexistence of Terrestrial and Aerial Users in Cellular Networks

Cited by in corpus (37)

• Communication and Control in Collaborative UAVs: Recent Advances and Future Trends
• An Extensive Survey on the Internet of Drones
• A Survey of Wireless Networks for Future Aerial COMmunications (FACOM)
• Learning-Based UAV Path Planning for Data Collection with Integrated Collision Avoidance
• Artificial Intelligence for UAV-enabled Wireless Networks: A Survey
• Cognitive UAV Communication via Joint Maneuver and Power Control
• Deep Reinforcement Learning for Trajectory Path Planning and Distributed Inference in Resource-Constrained UAV Swarms
• A Survey on Reservoir Computing and its Interdisciplinary Applications Beyond Traditional Machine Learning
• Non-Orthogonal Multiple Access for Air-to-Ground Communication
• Distributed CNN Inference on Resource-Constrained UAVs for Surveillance Systems: Design and Optimization
• Cellular-Connected UAV with Adaptive Air-to-Ground Interference Cancellation and Trajectory Optimization
• Deep Reinforcement Learning for Autonomous Internet of Things: Model, Applications and Challenges
• Radio Map Assisted Approach for Interference-Aware Predictive UAV Communications
• Radio Map Assisted Routing and Predictive Resource Allocation over Dynamic Low Altitude Networks
• Explainable Artificial Intelligence (XAI) for 6G: Improving Trust between Human and Machine
• Simultaneous Navigation and Radio Mapping for Cellular-Connected UAV with Deep Reinforcement Learning
• Radio Map Based 3D Path Planning for Cellular-Connected UAV
• Distributed Multi-agent Meta Learning for Trajectory Design in Wireless Drone Networks
• Path Design for Cellular-Connected UAV with Reinforcement Learning
• A Deep Reinforcement Learning Approach to Efficient Drone Mobility Support
• Hybrid Offline-Online Design for UAV-Enabled Data Harvesting in Probabilistic LoS Channel
• Uncrewed Vehicles in 6G Networks: A Unifying Treatment of Problems, Formulations, and Tools
• Learning-Based UAV Trajectory Optimization with Collision Avoidance and Connectivity Constraints
• Adaptive Height Optimisation for Cellular-Connected UAVs using Reinforcement Learning
• UAV-to-Device Underlay Communications: Age of Information Minimization by Multi-agent Deep Reinforcement Learning
• Channel State Acquisition in Uplink NOMA for Cellular-Connected UAV: Exploitation of Doppler and Modulation Diversities
• Edge Computing Assisted Autonomous Flight for UAV: Synergies between Vision and Communications
• Reinforcement Learning Based Dynamic Power Control for UAV Mobility Management
• Online Maneuver Design for UAV-Enabled NOMA Systems via Reinforcement Learning
• Intelligent Trajectory Planning in UAV-mounted Wireless Networks: A Quantum-Inspired Reinforcement Learning Perspective
• 3D-Map Assisted UAV Trajectory Design Under Cellular Connectivity Constraints
• Cellular UAV-to-Device Communications: Trajectory Design and Mode Selection by Multi-agent Deep Reinforcement Learning
• Two-Hop Multi-UAV Relay Network Optimization with Directional Antennas
• Voting-Based Multi-Agent Reinforcement Learning for Intelligent IoT
• Joint Mobile Charging and Coverage-Time Extension for Unmanned Aerial Vehicles
• Responsive Regulation of Dynamic UAV Communication Networks Based on Deep Reinforcement Learning
• Reinforcement Learning-based Joint Path and Energy Optimization of Cellular-Connected Unmanned Aerial Vehicles