Reinforcement-learning-based control of convectively-unstable flows

Abstract

This work reports the application of a model-free deep-reinforcement-learning-based (DRL) flow control strategy to suppress perturbations evolving in the 1-D linearised Kuramoto-Sivashinsky (KS) equation and 2-D boundary layer flows. The former is commonly used to model the disturbance developing in flat-plate boundary layer flows. These flow systems are convectively unstable, being able to amplify the upstream disturbance, and are thus difficult to control. The control action is implemented through a volumetric force at a fixed position and the control performance is evaluated by the reduction of perturbation amplitude downstream. We first demonstrate the effectiveness of the DRL-based control in the KS system subjected to a random upstream noise. The amplitude of perturbation monitored downstream is significantly reduced and the learnt policy is shown to be robust to both measurement and external noise. One of our focuses is to optimally place sensors in the DRL control using the gradient-free particle swarm optimisation algorithm. After the optimisation process for different numbers of sensors, a specific eight-sensor placement is found to yield the best control performance. The optimised sensor placement in the KS equation is applied directly to control 2-D Blasius boundary layer flows and can efficiently reduce the downstream perturbation energy. Via flow analyses, the control mechanism found by DRL is the opposition control. Besides, it is found that when the flow instability information is embedded in the reward function of DRL to penalise the instability, the control performance can be further improved in this convectively-unstable flow.

References in corpus (11)

• Deep Learning in Neural Networks: An Overview
• Closed-loop separation control using machine learning
• Robust active flow control over a range of Reynolds numbers using an artificial neural network trained through deep reinforcement learning
• A review on deep reinforcement learning for fluid mechanics: an update
• Reinforcement-learning-based control of confined cylinder wakes with stability analyses
• Reinforcement Learning of Control Strategies for Reducing Skin Friction Drag in a Fully Developed Channel Flow
• Comparative analysis of machine learning methods for active flow control
• Deep reinforcement learning for large-eddy simulation modeling in wall-bounded turbulence
• Machine learning flow control with few sensor feedback and measurement noise
• Data-driven control of spatiotemporal chaos with reduced-order neural ODE-based models and reinforcement learning
• Optimal sensor and actuator placement for feedback control of vortex shedding

Cited by in corpus (5)

• Recent advances in applying deep reinforcement learning for flow control: perspectives and future directions
• Effective control of two-dimensional Rayleigh--Bénard convection: invariant multi-agent reinforcement learning is all you need
• How to Control Hydrodynamic Force on Fluidic Pinball via Deep Reinforcement Learning
• Deep reinforcement learning for tracking a moving target in jellyfish-like swimming
• Reinforcement-learning-assisted control of four-roll mills: geometric symmetry and inertial effect