Two-Timescale Voltage Control in Distribution Grids Using Deep Reinforcement Learning

Abstract

Modern distribution grids are currently being challenged by frequent and sizable voltage fluctuations, due mainly to the increasing deployment of electric vehicles and renewable generators. Existing approaches to maintaining bus voltage magnitudes within the desired region can cope with either traditional utility-owned devices (e.g., shunt capacitors), or contemporary smart inverters that come with distributed generation units (e.g., photovoltaic plants). The discrete on-off commitment of capacitor units is often configured on an hourly or daily basis, yet smart inverters can be controlled within milliseconds, thus challenging joint control of these two types of assets. In this context, a novel two-timescale voltage regulation scheme is developed for distribution grids by judiciously coupling data-driven with physicsbased optimization. On a faster timescale, say every second, the optimal setpoints of smart inverters are obtained by minimizing instantaneous bus voltage deviations from their nominal values, based on either the exact alternating current power flow model or a linear approximant of it; whereas, on the slower timescale (e.g., every hour), shunt capacitors are configured to minimize the longterm discounted voltage deviations using a deep reinforcement learning algorithm. Extensive numerical tests on a real-world 47- bus distribution network as well as the IEEE 123-bus test feeder using real data corroborate the effectiveness of the novel scheme.

arXiv comment10 pages, 12 figures, to appear in the IEEE Transactions on Smart Grid

References in corpus (2)

• Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation
• Real-time Power System State Estimation and Forecasting via Deep Neural Networks

Cited by in corpus (8)

• Learning an Optimally Reduced Formulation of OPF through Meta-optimization
• Finite-Sample Analysis of Decentralized Temporal-Difference Learning with Linear Function Approximation
• Model-Free Voltage Regulation of Unbalanced Distribution Network Based on Surrogate Model and Deep Reinforcement Learning
• Bi-level Off-policy Reinforcement Learning for Volt/VAR Control Involving Continuous and Discrete Devices
• Action Set Based Policy Optimization for Safe Power Grid Management
• Data-Driven Secondary Control of Distributed Energy Resources
• Learning Connectivity and Higher-order Interactions in Radial Distribution Grids
• Robust, Deep, and Reinforcement Learning for Management of Communication and Power Networks