Applying deep reinforcement learning to active flow control in turbulent conditions

Abstract

Machine learning has recently become a promising technique in fluid mechanics, especially for active flow control (AFC) applications. A recent work [J. Fluid Mech. (2019), vol. 865, pp. 281-302] has demonstrated the feasibility and effectiveness of deep reinforcement learning (DRL) in performing AFC over a circular cylinder at $Re=100$, i.e., in the laminar flow regime. As a follow-up study, we investigate the same AFC problem at an intermediate Reynolds number, i.e., $Re=1000$, where the turbulence in the flow poses great challenges to the control. The results show that the DRL agent can still find effective control strategies, but requires much more episodes in the learning. A remarkable drag reduction of around $30\%$ is achieved, which is accompanied by elongation of the recirculation bubble and reduction of turbulent fluctuations in the cylinder wake. To our best knowledge, this study is the first successful application of DRL to AFC in weak turbulent conditions. It therefore sets a new milestone in progressing towards AFC in strong turbulent flows.

References in corpus (11)

• Adam: A Method for Stochastic Optimization
• Machine Learning for Fluid Mechanics
• Emergence of Locomotion Behaviours in Rich Environments
• Artificial Neural Networks trained through Deep Reinforcement Learning discover control strategies for active flow control
• Accelerating Deep Reinforcement Learning strategies of Flow Control through a multi-environment approach
• Closed-loop separation control using machine learning
• Robust active flow control over a range of Reynolds numbers using an artificial neural network trained through deep reinforcement learning
• Artificial intelligence control of a turbulent jet
• Low-order model for successive bifurcations of the fluidic pinball
• Controlling Rayleigh-Bénard convection via Reinforcement Learning
• Control of chaotic systems by Deep Reinforcement Learning

Cited by in corpus (22)

• Physics-informed neural networks for solving Reynolds-averaged Navier$\unicode{x2013}$Stokes equations
• Recent advances in applying deep reinforcement learning for flow control: perspectives and future directions
• A review on Deep Reinforcement Learning for Fluid Mechanics
• A review on deep reinforcement learning for fluid mechanics: an update
• Experimental velocity data estimation for imperfect particle images using machine learning
• Reinforcement Learning of Control Strategies for Reducing Skin Friction Drag in a Fully Developed Channel Flow
• DRLinFluids -- An open-source python platform of coupling Deep Reinforcement Learning and OpenFOAM
• Dynamic Feature-based Deep Reinforcement Learning for Flow Control of Circular Cylinder with Sparse Surface Pressure Sensing
• Comparative analysis of machine learning methods for active flow control
• Bluff body uses deep-reinforcement-learning trained active flow control to achieve hydrodynamic stealth
• Effective control of two-dimensional Rayleigh--Bénard convection: invariant multi-agent reinforcement learning is all you need
• Machine learning flow control with few sensor feedback and measurement noise
• Reinforcement-learning-based actuator selection method for active flow control
• Reinforcement-learning-based control of convectively-unstable flows
• Data-driven control of spatiotemporal chaos with reduced-order neural ODE-based models and reinforcement learning
• Stabilization of a multi-frequency open cavity flow with gradient-enriched machine learning control
• Deep learning delay coordinate dynamics for chaotic attractors from partial observable data
• Deep reinforcement learning for the management of the wall regeneration cycle in wall-bounded turbulent flows
• How to Control Hydrodynamic Force on Fluidic Pinball via Deep Reinforcement Learning
• Model Predictive and Reinforcement Learning Methods for Active Flow Control of an Airfoil with Dual-point Excitation of Plasma Actuators
• Physics-informed Dyna-Style Model-Based Deep Reinforcement Learning for Dynamic Control
• Reinforcement-learning-assisted control of four-roll mills: geometric symmetry and inertial effect