HUGO -- Highlighting Unseen Grid Options: Combining Deep Reinforcement Learning with a Heuristic Target Topology Approach

Abstract

With the growth of Renewable Energy (RE) generation, the operation of power grids has become increasingly complex. One solution could be automated grid operation, where Deep Reinforcement Learning (DRL) has repeatedly shown significant potential in Learning to Run a Power Network (L2RPN) challenges. However, only individual actions at the substation level have been subjected to topology optimization by most existing DRL algorithms. In contrast, we propose a more holistic approach by proposing specific Target Topologies (TTs) as actions. These topologies are selected based on their robustness. As part of this paper, we present a search algorithm to find the TTs and upgrade our previously developed DRL agent CurriculumAgent (CAgent) to a novel topology agent. We compare the upgrade to the previous CAgent and can increase their L2RPN score significantly by 10%. Further, we achieve a 25% better median survival time with our TTs included. Later analysis shows that almost all TTs are close to the base topology, explaining their robustness

12 pages + 2 pages references, 9 Figures, submission planed in Sustainable Energy, Grids and Networks

References in corpus (11)

• Mastering Atari, Go, Chess and Shogi by Planning with a Learned Model
• pandapower - an Open Source Python Tool for Convenient Modeling, Analysis and Optimization of Electric Power Systems
• Learning Quadrupedal Locomotion over Challenging Terrain
• Mastering Chess and Shogi by Self-Play with a General Reinforcement Learning Algorithm
• Dota 2 with Large Scale Deep Reinforcement Learning
• Population Based Training of Neural Networks
• Learning to run a power network challenge for training topology controllers
• Managing power grids through topology actions: A comparative study between advanced rule-based and reinforcement learning agents
• Power Grid Congestion Management via Topology Optimization with AlphaZero
• Multi-Agent Reinforcement Learning for Power Grid Topology Optimization
• Hierarchical Reinforcement Learning for Power Network Topology Control

Cited by in corpus (3)

• Graph Reinforcement Learning for Power Grids: A Comprehensive Survey
• Learning Topology Actions for Power Grid Control: A Graph-Based Soft-Label Imitation Learning Approach
• Centrally Coordinated Multi-Agent Reinforcement Learning for Power Grid Topology Control