Proximal Policy Optimization-based Transmit Beamforming and Phase-shift Design in an IRS-aided ISAC System for the THz Band

Abstract

In this paper, an IRS-aided integrated sensing and communications (ISAC) system operating in the terahertz (THz) band is proposed to maximize the system capacity. Transmit beamforming and phase-shift design are transformed into a universal optimization problem with ergodic constraints. Then the joint optimization of transmit beamforming and phase-shift design is achieved by gradient-based, primal-dual proximal policy optimization (PPO) in the multi-user multiple-input single-output (MISO) scenario. Specifically, the actor part generates continuous transmit beamforming and the critic part takes charge of discrete phase shift design. Based on the MISO scenario, we investigate a distributed PPO (DPPO) framework with the concept of multi-threading learning in the multi-user multiple-input multiple-output (MIMO) scenario. Simulation results demonstrate the effectiveness of the primal-dual PPO algorithm and its multi-threading version in terms of transmit beamforming and phase-shift design.

References in corpus (7)

• Deep Reinforcement Learning for Online Computation Offloading in Wireless Powered Mobile-Edge Computing Networks
• Joint Transmit Beamforming for Multiuser MIMO Communication and MIMO Radar
• Deep Reinforcement Learning Based Intelligent Reflecting Surface for Secure Wireless Communications
• Deep Reinforcement Learning for 5G Networks: Joint Beamforming, Power Control, and Interference Coordination
• Learning Optimal Resource Allocations in Wireless Systems
• Optimal and Scalable Caching for 5G Using Reinforcement Learning of Space-time Popularities
• DeepPool: Distributed Model-free Algorithm for Ride-sharing using Deep Reinforcement Learning

Cited by in corpus (5)

• Joint Beamforming for RIS-Assisted Integrated Sensing and Communication Systems
• A Survey of Predictive Maintenance: Systems, Purposes and Approaches
• Joint Communication and Sensing in RIS-enabled mmWave Networks
• Physical Layer Security for Integrated Sensing and Communication: A Survey
• Beamforming and Phase Shift Design for STAR-RIS Assisted Secure Sensing and Communication in ISAC Systems