Robust active flow control over a range of Reynolds numbers using an artificial neural network trained through deep reinforcement learning

Abstract

This paper focuses on the active flow control of a computational fluid dynamics simulation over a range of Reynolds numbers using deep reinforcement learning (DRL). More precisely, the proximal policy optimization (PPO) method is used to control the mass flow rate of four synthetic jets symmetrically located on the upper and lower sides of a cylinder immersed in a two-dimensional flow domain. The learning environment supports four flow configurations with Reynolds numbers 100, 200, 300 and 400, respectively. A new smoothing interpolation function is proposed to help the PPO algorithm to learn to set continuous actions, which is of great importance to effectively suppress problematic jumps in lift and allow a better convergence for the training process. It is shown that the DRL controller is able to significantly reduce the lift and drag fluctuations and to actively reduce the drag by approximately 5.7%, 21.6%, 32.7%, and 38.7%, at $Re$=100, 200, 300, and 400 respectively. More importantly, it can also effectively reduce drag for any previously unseen value of the Reynolds number between 60 and 400. This highlights the generalization ability of deep neural networks and is an important milestone to active flow control.

References in corpus (21)

• Playing Atari with Deep Reinforcement Learning
• The role of artificial intelligence in achieving the Sustainable Development Goals
• Machine Learning for Fluid Mechanics
• DGM: A deep learning algorithm for solving partial differential equations
• Benchmarking Deep Reinforcement Learning for Continuous Control
• Deep Learning Methods for Reynolds-Averaged Navier-Stokes Simulations of Airfoil Flows
• Artificial Neural Networks trained through Deep Reinforcement Learning discover control strategies for active flow control
• Deep Reinforcement Learning for Dialogue Generation
• Deep Neural Networks for Data-Driven Turbulence Models
• Continuous Deep Q-Learning with Model-based Acceleration
• Flow Navigation by Smart Microswimmers via Reinforcement Learning
• Deep Reinforcement Learning Attitude Control of Fixed-Wing UAVs Using Proximal Policy Optimization
• Closed-loop separation control using machine learning
• Accelerating Deep Reinforcement Learning strategies of Flow Control through a multi-environment approach
• Multi-Scale Proper Orthogonal Decomposition of Complex Fluid Flows
• Learning to school in the presence of hydrodynamic interactions
• Cormorant: Covariant Molecular Neural Networks
• Data-driven recovery of hidden physics in reduced order modeling of fluid flows
• Closed-loop separation control over a sharp edge ramp using Genetic Programming
• Optimal control of circular cylinder wakes using long control horizons
• Adaptive Skip Intervals: Temporal Abstraction for Recurrent Dynamical Models

Cited by in corpus (27)

• Physics-informed neural networks for solving Reynolds-averaged Navier$\unicode{x2013}$Stokes equations
• Deep Neural Networks for Nonlinear Model Order Reduction of Unsteady Flows
• Applying deep reinforcement learning to active flow control in turbulent conditions
• Recent advances in applying deep reinforcement learning for flow control: perspectives and future directions
• A Perspective on Machine Learning Methods in Turbulence Modelling
• A review on Deep Reinforcement Learning for Fluid Mechanics
• A review on deep reinforcement learning for fluid mechanics: an update
• Reinforcement-learning-based control of confined cylinder wakes with stability analyses
• Experimental velocity data estimation for imperfect particle images using machine learning
• Generalization techniques of neural networks for fluid flow estimation
• Deep Reinforcement Learning for Turbulence Modeling in Large Eddy Simulations
• Reinforcement Learning of Control Strategies for Reducing Skin Friction Drag in a Fully Developed Channel Flow
• DRLinFluids -- An open-source python platform of coupling Deep Reinforcement Learning and OpenFOAM
• Dynamic Feature-based Deep Reinforcement Learning for Flow Control of Circular Cylinder with Sparse Surface Pressure Sensing
• Comparative analysis of machine learning methods for active flow control
• Bluff body uses deep-reinforcement-learning trained active flow control to achieve hydrodynamic stealth
• Effective control of two-dimensional Rayleigh--Bénard convection: invariant multi-agent reinforcement learning is all you need
• Deep Reinforcement Learning for Computational Fluid Dynamics on HPC Systems
• Deep reinforcement learning for large-eddy simulation modeling in wall-bounded turbulence
• Machine learning flow control with few sensor feedback and measurement noise
• Stabilization of the fluidic pinball with gradient-enriched machine learning control
• Multi-fidelity reinforcement learning framework for shape optimization
• Reinforcement-learning-based control of convectively-unstable flows
• Discovering explicit Reynolds-averaged turbulence closures for turbulent separated flows through deep learning-based symbolic regression with non-linear corrections
• Toward Discretization-Consistent Closure Schemes for Large Eddy Simulation Using Reinforcement Learning
• Deep reinforcement learning for the management of the wall regeneration cycle in wall-bounded turbulent flows
• An Embedded Feature Selection Framework for Control