Log-law recovery through reinforcement-learning wall model for large-eddy simulation

Abstract

This paper focuses on the use of reinforcement learning (RL) as a machine-learning (ML) modeling tool for near-wall turbulence. RL has demonstrated its effectiveness in solving high-dimensional problems, especially in domains such as games. Despite its potential, RL is still not widely used for turbulence modeling and is primarily used for flow control and optimization purposes. A new RL wall model (WM) called VYBA23 is developed in this work, which uses agents dispersed in the flow near the wall. The model is trained on a single Reynolds number ($R{e}_{\tau }=10^4$) and does not rely on high-fidelity data, as the back-propagation process is based on a reward rather than output error. The states of the RLWM, which are the representation of the environment by the agents, are normalized to remove dependence on the Reynolds number. The model is tested and compared to another RLWM (BK22) and to an equilibrium wall model, in a half-channel flow at eleven different Reynolds numbers ($R{e}_{\tau }\in [180;10^{10}]$). The effects of varying agents' parameters such as actions range, time-step, and spacing are also studied. The results are promising, showing little effect on the average flow field but some effect on wall-shear stress fluctuations and velocity fluctuations. This work offers positive prospects for developing RLWMs that can recover physical laws, and for extending this type of ML models to more complex flows in the future.

arXiv commentarXiv admin note: text overlap with arXiv:2211.03614

References in corpus (23)

• Machine Learning for Fluid Mechanics
• Turbulence Modeling in the Age of Data
• Direct numerical simulation of turbulent channel flow up to $R{e}_{\tau }\approx 5200$
• Deep Reinforcement Learning for Cyber Security
• Enhancing Computational Fluid Dynamics with Machine Learning
• A Comprehensive Physics-Informed Machine Learning Framework for Predictive Turbulence Modeling
• Physics-Informed Machine Learning Approach for Augmenting Turbulence Models: A Comprehensive Framework
• Perspectives on Machine Learning-augmented Reynolds-averaged and Large Eddy Simulation Models of Turbulence
• Machine-learning-based reduced order modeling for unsteady flows around bluff bodies of various shapes
• Training convolutional neural networks to estimate turbulent sub-grid scale reaction rates
• Scientific multi-agent reinforcement learning for wall-models of turbulent flows
• Dynamic slip wall model for large-eddy simulation
• Move Evaluation in Go Using Deep Convolutional Neural Networks
• Learning dominant physical processes with data-driven balance models
• Sub-grid scale model classification and blending through deep learning
• Deep Reinforcement Learning for Turbulence Modeling in Large Eddy Simulations
• Deep reinforcement learning for large-eddy simulation modeling in wall-bounded turbulence
• A Lagrangian relaxation towards equilibrium wall model for large eddy simulation
• Self-critical machine-learning wall-modeled LES for external aerodynamics
• Inserting machine-learned virtual wall velocity for large-eddy simulation of turbulent channel flows
• Remember and Forget for Experience Replay
• POD-mode-augmented wall model and its applications to flows at non-equilibrium conditions
• Discretization-independent surrogate modeling over complex geometries using hypernetworks and implicit representations

Cited by in corpus (3)

• Toward Discretization-Consistent Closure Schemes for Large Eddy Simulation Using Reinforcement Learning
• Ensemble data assimilation-based mixed subgrid-scale model for large-eddy simulations
• Large Language Model Driven Development of Turbulence Models