Recent advances in applying deep reinforcement learning for flow control: perspectives and future directions

Abstract

Deep reinforcement learning (DRL) has been applied to a variety of problems during the past decade, and has provided effective control strategies in high-dimensional and non-linear situations that are challenging to traditional methods. Flourishing applications now spread out into the field of fluid dynamics, and specifically of active flow control (AFC). In the community of AFC, the encouraging results obtained in two-dimensional and chaotic conditions have raised interest to study increasingly complex flows. In this review, we first provide a general overview of the reinforcement-learning (RL) and DRL frameworks, as well as their recent advances. We then focus on the application of DRL to AFC, highlighting the current limitations of the DRL algorithms in this field, and suggesting some of the potential upcoming milestones to reach, as well as open questions that are likely to attract the attention of the fluid-mechanics community.

References in corpus (45)

• Continuous control with deep reinforcement learning
• Machine Learning for Fluid Mechanics
• Addressing Function Approximation Error in Actor-Critic Methods
• Dueling Network Architectures for Deep Reinforcement Learning
• Asynchronous Methods for Deep Reinforcement Learning
• DeepMimic: Example-Guided Deep Reinforcement Learning of Physics-Based Character Skills
• Enhancing Computational Fluid Dynamics with Machine Learning
• Artificial Neural Networks trained through Deep Reinforcement Learning discover control strategies for active flow control
• Deep Reinforcement Learning for Dialogue Generation
• Massively Parallel Methods for Deep Reinforcement Learning
• Deep Neural Networks for Data-Driven Turbulence Models
• Reinforcement Learning with Unsupervised Auxiliary Tasks
• Closed-loop separation control using machine learning
• Accelerating Deep Reinforcement Learning strategies of Flow Control through a multi-environment approach
• Robust active flow control over a range of Reynolds numbers using an artificial neural network trained through deep reinforcement learning
• Fish schooling as a basis for vertical axis wind turbine farm design
• Deep Reinforcement Learning in a Handful of Trials using Probabilistic Dynamics Models
• Least Squares Shadowing sensitivity analysis of chaotic limit cycle oscillations
• Scientific multi-agent reinforcement learning for wall-models of turbulent flows
• Applying deep reinforcement learning to active flow control in turbulent conditions
• Synchronised Swimming of Two Fish
• A Perspective on Machine Learning Methods in Turbulence Modelling
• Learning to school in the presence of hydrodynamic interactions
• Equivariance Through Parameter-Sharing
• A review on deep reinforcement learning for fluid mechanics: an update
• A review on Deep Reinforcement Learning for Fluid Mechanics
• Reinforcement-learning-based control of confined cylinder wakes with stability analyses
• Drag reduction of a car model by linear genetic programming control
• Controlling Rayleigh-Bénard convection via Reinforcement Learning
• Deep Reinforcement Learning for Turbulence Modeling in Large Eddy Simulations
• Diffusive dynamics and stochastic models of turbulent axisymmetric wakes
• Control of chaotic systems by Deep Reinforcement Learning
• Reinforcement Learning of Control Strategies for Reducing Skin Friction Drag in a Fully Developed Channel Flow
• DRLinFluids -- An open-source python platform of coupling Deep Reinforcement Learning and OpenFOAM
• Comparative analysis of machine learning methods for active flow control
• Controlling the onset of turbulence by streamwise traveling waves. Part 2. Direct numerical simulations
• Bluff body uses deep-reinforcement-learning trained active flow control to achieve hydrodynamic stealth
• Deep Reinforcement Learning for Computational Fluid Dynamics on HPC Systems
• Learning to swim in potential flow
• Universal approximations of permutation invariant/equivariant functions by deep neural networks
• Relative Entropy Regularized Policy Iteration
• Optimal control of circular cylinder wakes using long control horizons
• Reinforcement-learning-based actuator selection method for active flow control
• Reinforcement-learning-based control of convectively-unstable flows
• An application of data driven reward of deep reinforcement learning by dynamic mode decomposition in active flow control

Cited by in corpus (11)

• Long-term predictions of turbulence by implicit U-Net enhanced Fourier neural operator
• Dynamic Feature-based Deep Reinforcement Learning for Flow Control of Circular Cylinder with Sparse Surface Pressure Sensing
• Effective control of two-dimensional Rayleigh--Bénard convection: invariant multi-agent reinforcement learning is all you need
• Toward Discretization-Consistent Closure Schemes for Large Eddy Simulation Using Reinforcement Learning
• Latent feedback control of distributed systems in multiple scenarios through deep learning-based reduced order models
• Real-time optimal control of high-dimensional parametrized systems by deep learning-based reduced order models
• Deep reinforcement learning for the management of the wall regeneration cycle in wall-bounded turbulent flows
• Deep reinforcement learning for tracking a moving target in jellyfish-like swimming
• Model Predictive and Reinforcement Learning Methods for Active Flow Control of an Airfoil with Dual-point Excitation of Plasma Actuators
• Prediction of flow and elastic stresses in a viscoelastic turbulent channel flow using convolutional neural networks
• Data-assimilated model-informed reinforcement learning