Bridging Transient and Steady-State Performance in Voltage Control: A Reinforcement Learning Approach with Safe Gradient Flow

Abstract

Deep reinforcement learning approaches are becoming appealing for the design of nonlinear controllers for voltage control problems, but the lack of stability guarantees hinders their deployment in real-world scenarios. This paper constructs a decentralized RL-based controller featuring two components: a transient control policy and a steady-state performance optimizer. The transient policy is parameterized as a neural network, and the steady-state optimizer represents the gradient of the long-term operating cost function. The two parts are synthesized through a safe gradient flow framework, which prevents the violation of reactive power capacity constraints. We prove that if the output of the transient controller is bounded and monotonically decreasing with respect to its input, then the closed-loop system is asymptotically stable and converges to the optimal steady-state solution. We demonstrate the effectiveness of our method by conducting experiments with IEEE 13-bus and 123-bus distribution system test feeders.

arXiv commentPublished in IEEE Control Systems Letters, vol. 7, pp. 2845-2850, 2023 with CDC presentation

References in corpus (8)

• Trust Region Policy Optimization
• pandapower - an Open Source Python Tool for Convenient Modeling, Analysis and Optimization of Electric Power Systems
• Options for Control of Reactive Power by Distributed Photovoltaic Generators
• Reinforcement Learning for Selective Key Applications in Power Systems: Recent Advances and Future Challenges
• Optimal Distributed Feedback Voltage Control under Limited Reactive Power
• Control Barrier Function Based Design of Gradient Flows for Constrained Nonlinear Programming
• Learning Provably Stable Local Volt/Var Controllers for Efficient Network Operation
• Structured Neural-PI Control for Networked Systems: Stability and Steady-State Optimality Guarantees

Cited by in corpus (2)

• A Review of Safe Reinforcement Learning Methods for Modern Power Systems
• Unsupervised Learning for Equitable DER Control