Deep Reinforcement Learning for Computational Fluid Dynamics on HPC Systems

Abstract

Reinforcement learning (RL) is highly suitable for devising control strategies in the context of dynamical systems. A prominent instance of such a dynamical system is the system of equations governing fluid dynamics. Recent research results indicate that RL-augmented computational fluid dynamics (CFD) solvers can exceed the current state of the art, for example in the field of turbulence modeling. However, while in supervised learning, the training data can be generated a priori in an offline manner, RL requires constant run-time interaction and data exchange with the CFD solver during training. In order to leverage the potential of RL-enhanced CFD, the interaction between the CFD solver and the RL algorithm thus have to be implemented efficiently on high-performance computing (HPC) hardware. To this end, we present Relexi as a scalable RL framework that bridges the gap between machine learning workflows and modern CFD solvers on HPC systems providing both components with its specialized hardware. Relexi is built with modularity in mind and allows easy integration of various HPC solvers by means of the in-memory data transfer provided by the SmartSim library. Here, we demonstrate that the Relexi framework can scale up to hundreds of parallel environment on thousands of cores. This allows to leverage modern HPC resources to either enable larger problems or faster turnaround times. Finally, we demonstrate the potential of an RL-augmented CFD solver by finding a control strategy for optimal eddy viscosity selection in large eddy simulations.

11 pages, 5 figures, the code can be accessed at https://github.com/flexi-framework/relexi

References in corpus (7)

• Robust active flow control over a range of Reynolds numbers using an artificial neural network trained through deep reinforcement learning
• Scientific multi-agent reinforcement learning for wall-models of turbulent flows
• A Perspective on Machine Learning Methods in Turbulence Modelling
• Reinforcement-learning-based control of confined cylinder wakes with stability analyses
• Deep Reinforcement Learning for Turbulence Modeling in Large Eddy Simulations
• A machine learning framework for LES closure terms
• Using Machine Learning at Scale in HPC Simulations with SmartSim: An Application to Ocean Climate Modeling

Cited by in corpus (6)

• Recent advances in applying deep reinforcement learning for flow control: perspectives and future directions
• Deep Reinforcement Learning for Turbulence Modeling in Large Eddy Simulations
• Effective control of two-dimensional Rayleigh--Bénard convection: invariant multi-agent reinforcement learning is all you need
• Toward Discretization-Consistent Closure Schemes for Large Eddy Simulation Using Reinforcement Learning
• HPC-based Solvers of Minimisation Problems for Signal Processing
• Assessment of jet inflow conditions on the development of supersonic jet flows