Federated Deep Reinforcement Learning for RIS-Assisted Indoor Multi-Robot Communication Systems

Abstract

Indoor multi-robot communications face two key challenges: one is the severe signal strength degradation caused by blockages (e.g., walls) and the other is the dynamic environment caused by robot mobility. To address these issues, we consider the reconfigurable intelligent surface (RIS) to overcome the signal blockage and assist the trajectory design among multiple robots. Meanwhile, the non-orthogonal multiple access (NOMA) is adopted to cope with the scarcity of spectrum and enhance the connectivity of robots. Considering the limited battery capacity of robots, we aim to maximize the energy efficiency by jointly optimizing the transmit power of the access point (AP), the phase shifts of the RIS, and the trajectory of robots. A novel federated deep reinforcement learning (F-DRL) approach is developed to solve this challenging problem with one dynamic long-term objective. Through each robot planning its path and downlink power, the AP only needs to determine the phase shifts of the RIS, which can significantly save the computation overhead due to the reduced training dimension. Simulation results reveal the following findings: I) the proposed F-DRL can reduce at least 86% convergence time compared to the centralized DRL; II) the designed algorithm can adapt to the increasing number of robots; III) compared to traditional OMA-based benchmarks, NOMA-enhanced schemes can achieve higher energy efficiency.

arXiv commentThe paper has been accepted for publication in the IEEE Transactions on Vehicular Technology

References in corpus (7)

• Deep Reinforcement Learning for Multi-Agent Systems: A Review of Challenges, Solutions and Applications
• Artificial Intelligence-Enabled Intelligent 6G Networks
• Deep Reinforcement Learning Based Intelligent Reflecting Surface for Secure Wireless Communications
• Resource Allocation and Service Provisioning in Multi-Agent Cloud Robotics: A Comprehensive Survey
• STAR-RIS Integrated Non-Orthogonal Multiple Access and Over-the-Air Federated Learning: Framework, Analysis, and Optimization
• Integrating Over-the-Air Federated Learning and Non-Orthogonal Multiple Access: What Role can RIS Play?
• Intelligent Reflecting Surface Enhanced Indoor Robot Path Planning: A Radio Map based Approach

Cited by in corpus (2)

• Design Optimization of NOMA Aided Multi-STAR-RIS for Indoor Environments: A Convex Approximation Imitated Reinforcement Learning Approach
• Reconfigurable Intelligent Surface for Internet of Robotic Things