Deep Reinforcement Learning for Turbulence Modeling in Large Eddy Simulations

Abstract

Over the last years, supervised learning (SL) has established itself as the state-of-the-art for data-driven turbulence modeling. In the SL paradigm, models are trained based on a dataset, which is typically computed a priori from a high-fidelity solution by applying the respective filter function, which separates the resolved and the unresolved flow scales. For implicitly filtered large eddy simulation (LES), this approach is infeasible, since here, the employed discretization itself acts as an implicit filter function. As a consequence, the exact filter form is generally not known and thus, the corresponding closure terms cannot be computed even if the full solution is available. The reinforcement learning (RL) paradigm can be used to avoid this inconsistency by training not on a previously obtained training dataset, but instead by interacting directly with the dynamical LES environment itself. This allows to incorporate the potentially complex implicit LES filter into the training process by design. In this work, we apply a reinforcement learning framework to find an optimal eddy-viscosity for implicitly filtered large eddy simulations of forced homogeneous isotropic turbulence. For this, we formulate the task of turbulence modeling as an RL task with a policy network based on convolutional neural networks that adapts the eddy-viscosity in LES dynamically in space and time based on the local flow state only. We demonstrate that the trained models can provide long-term stable simulations and that they outperform established analytical models in terms of accuracy. In addition, the models generalize well to other resolutions and discretizations. We thus demonstrate that RL can provide a framework for consistent, accurate and stable turbulence modeling especially for implicitly filtered LES.

17 pages, 9 figures. Accepted Manuscript

References in corpus (5)

• Robust active flow control over a range of Reynolds numbers using an artificial neural network trained through deep reinforcement learning
• Learning physics-constrained subgrid-scale closures in the small-data regime for stable and accurate LES
• Deep reinforcement learning for large-eddy simulation modeling in wall-bounded turbulence
• Deep Reinforcement Learning for Computational Fluid Dynamics on HPC Systems
• Frame invariant neural network closures for Kraichnan turbulence

Cited by in corpus (17)

• Improving aircraft performance using machine learning: a review
• Recent advances in applying deep reinforcement learning for flow control: perspectives and future directions
• Deep Reinforcement Learning for Computational Fluid Dynamics on HPC Systems
• Log-law recovery through reinforcement-learning wall model for large-eddy simulation
• Discovering explicit Reynolds-averaged turbulence closures for turbulent separated flows through deep learning-based symbolic regression with non-linear corrections
• Toward Discretization-Consistent Closure Schemes for Large Eddy Simulation Using Reinforcement Learning
• Machine learning for online sea ice bias correction within global ice-ocean simulations
• Combining Machine Learning with Computational Fluid Dynamics using OpenFOAM and SmartSim
• Active flow control of a turbulent separation bubble through deep reinforcement learning
• A recursive neural-network-based subgrid-scale model for large eddy simulation: application to homogeneous isotropic turbulence
• Data-driven spectral turbulence modeling for Rayleigh-Bénard convection
• Adjoint-based variational optimal mixed models for large-eddy simulation of turbulence
• Kinetic data-driven approach to turbulence subgrid modeling
• Machine-learning-based simulation of turbulent flows over periodic hills using a hybrid U-Net and Fourier neural operator framework
• A new data-driven energy-stable Evolve-Filter-Relax model for turbulent flow simulation
• Addressing A Posteriori Performance Degradation in Neural Network Subgrid Stress Models
• Assessment of jet inflow conditions on the development of supersonic jet flows