Deep reinforcement learning for the management of the wall regeneration cycle in wall-bounded turbulent flows

Abstract

The wall cycle in wall-bounded turbulent flows is a complex turbulence regeneration mechanism that remains not fully understood. This study explores the potential of deep reinforcement learning (DRL) for managing the wall regeneration cycle to achieve desired flow dynamics. We integrate the StableBaselines3 DRL libraries with the open-source DNS solver CaNS to create a robust platform for dynamic flow control. The DRL agent interacts with the DNS environment, learning policies that modify wall boundary conditions to optimize objectives such as the reduction of the skin-friction coefficient or the enhancement of certain coherent structures features. Initial experiments demonstrate the capability of DRL to achieve drag-reduction rates comparable with those achieved via traditional methods, though limited to short time periods. We also propose a strategy to enhance the coherence of velocity streaks, assuming that maintaining straight streaks can inhibit instability and further reduce skin friction. The implementation makes use of the message-passing-interface (MPI) wrappers for efficient communication between the Python-based DRL agent and the DNS solver, ensuring scalability on high-performance computing architectures. Our results highlight the promise of DRL in flow control applications and underscore the need for more advanced control laws and objective functions. Future work will focus on optimizing actuation periods and exploring new computational architectures to extend the applicability and the efficiency of DRL in turbulent flow management.

References in corpus (9)

• Array Programming with NumPy
• PyTorch: An Imperative Style, High-Performance Deep Learning Library
• Robust active flow control over a range of Reynolds numbers using an artificial neural network trained through deep reinforcement learning
• Applying deep reinforcement learning to active flow control in turbulent conditions
• Recent advances in applying deep reinforcement learning for flow control: perspectives and future directions
• Reinforcement Learning of Control Strategies for Reducing Skin Friction Drag in a Fully Developed Channel Flow
• Effective control of two-dimensional Rayleigh--Bénard convection: invariant multi-agent reinforcement learning is all you need
• Data-driven control of spatiotemporal chaos with reduced-order neural ODE-based models and reinforcement learning
• Active flow control of a turbulent separation bubble through deep reinforcement learning