Reinforcement-learning-based control of confined cylinder wakes with stability analyses

Abstract

This work studies the application of a reinforcement-learning-based (RL) flow control strategy to the flow past a cylinder confined between two walls in order to suppress vortex shedding. The control action is blowing and suction of two synthetic jets on the cylinder. The theme of this study is to investigate how to use and embed physical information of the flow in the RL-based control. First, global linear stability and sensitivity analyses based on the time-mean flow and the steady flow (which is a solution to the Navier-Stokes equations) are conducted in a range of blockage ratios and Reynolds numbers. It is found that the most sensitive region in the wake extends itself when either parameter increases in the parameter range we investigated here. Then, we utilise these physical results to help design RL-based control policies. We find that the controlled wake converges to the unstable steady base flow, where the vortex shedding can be successfully suppressed. A persistent oscillating control seems necessary to maintain this unstable state. The RL algorithm is able to outperform a gradient-based optimisation method (optimised in a certain period of time) in the long run. Furthermore, when the flow stability information is embedded in the reward function to penalise the instability, the controlled flow may become more stable. Finally, according to the sensitivity analyses, the control is most efficient when the probes are placed in the most sensitive region. The control can be successful even when few probes are properly placed in this manner.

35 pages, 22 figures, accepted by JFM

References in corpus (4)

• An Introduction to Adjoint Problems
• Robust active flow control over a range of Reynolds numbers using an artificial neural network trained through deep reinforcement learning
• A review on deep reinforcement learning for fluid mechanics: an update
• Controlled reattachment in separated flows: a variational approach to recirculation length reduction

Cited by in corpus (19)

• Machine Learning in Aerodynamic Shape Optimization
• Recent advances in applying deep reinforcement learning for flow control: perspectives and future directions
• A review on deep reinforcement learning for fluid mechanics: an update
• Comparative analysis of machine learning methods for active flow control
• Effective control of two-dimensional Rayleigh--Bénard convection: invariant multi-agent reinforcement learning is all you need
• Deep Reinforcement Learning for Computational Fluid Dynamics on HPC Systems
• Machine learning flow control with few sensor feedback and measurement noise
• Reinforcement-learning-based actuator selection method for active flow control
• Reinforcement-learning-based control of convectively-unstable flows
• Data-driven control of spatiotemporal chaos with reduced-order neural ODE-based models and reinforcement learning
• Bio-inspired variable-stiffness flaps for hybrid flow control, tuned via reinforcement learning
• Genetically-inspired convective heat transfer enhancement in a turbulent boundary layer
• Reinforcement-learning-based control of turbulent channel flows at high Reynolds numbers
• Deep Reinforcement Learning for the Heat Transfer Control of Pulsating Impinging Jets
• How to Control Hydrodynamic Force on Fluidic Pinball via Deep Reinforcement Learning
• Deep reinforcement learning for tracking a moving target in jellyfish-like swimming
• Numerical analyses of wire-plate electrohydrodynamic flows
• Super-resolution reconstruction of turbulent flows from a single Lagrangian trajectory
• Reinforcement-learning-assisted control of four-roll mills: geometric symmetry and inertial effect