Reinforcement-learning-based actuator selection method for active flow control

Abstract

This paper addresses the issue of actuator selection for active flow control by proposing a novel method built on top of a reinforcement learning agent. Starting from a pre-trained agent using numerous actuators, the algorithm estimates the impact of a potential actuator removal on the value function, indicating the agent's performance. It is applied to two test cases, the one-dimensional Kuramoto-Sivashinsky equation and a laminar bi-dimensional flow around an airfoil at Re=1000 for different angles of attack ranging from 12 to 20 degrees, to demonstrate its capabilities and limits. The proposed actuator-sparsification method relies on a sequential elimination of the least relevant action components, starting from a fully developed layout. The relevancy of each component is evaluated using metrics based on the value function. Results show that, while still being limited by this intrinsic elimination paradigm (i.e. the sequential elimination), actuator patterns and obtained policies demonstrate relevant performances and allow to draw an accurate approximation of the Pareto front of performances versus actuator budget.

arXiv comment33 pages, 26 figures, submitted to Journal of Fluid Mechanics

References in corpus (17)

• Machine Learning for Fluid Mechanics
• Turbulence Modeling in the Age of Data
• Deep learning for universal linear embeddings of nonlinear dynamics
• Machine learning accelerated computational fluid dynamics
• A Physics Informed Machine Learning Approach for Reconstructing Reynolds Stress Modeling Discrepancies Based on DNS Data
• Artificial Neural Networks trained through Deep Reinforcement Learning discover control strategies for active flow control
• How to Train Your Robot with Deep Reinforcement Learning; Lessons We've Learned
• Prediction of Aerodynamic Flow Fields Using Convolutional Neural Networks
• Accelerating Deep Reinforcement Learning strategies of Flow Control through a multi-environment approach
• Machine-learning-based reduced order modeling for unsteady flows around bluff bodies of various shapes
• Resolvent-analysis-based design of airfoil separation control
• Applying deep reinforcement learning to active flow control in turbulent conditions
• Reinforcement-learning-based control of confined cylinder wakes with stability analyses
• Controlling Rayleigh-Bénard convection via Reinforcement Learning
• Control of chaotic systems by Deep Reinforcement Learning
• Optimal sensor and actuator placement for feedback control of vortex shedding
• Stabilising viscous extensional flows using Reinforcement Learning

Cited by in corpus (3)

• Recent advances in applying deep reinforcement learning for flow control: perspectives and future directions
• Effective control of two-dimensional Rayleigh--Bénard convection: invariant multi-agent reinforcement learning is all you need
• Model Predictive and Reinforcement Learning Methods for Active Flow Control of an Airfoil with Dual-point Excitation of Plasma Actuators