How to Control Hydrodynamic Force on Fluidic Pinball via Deep Reinforcement Learning

Abstract

Deep reinforcement learning (DRL) for fluidic pinball, three individually rotating cylinders in the uniform flow arranged in an equilaterally triangular configuration, can learn the efficient flow control strategies due to the validity of self-learning and data-driven state estimation for complex fluid dynamic problems. In this work, we present a DRL-based real-time feedback strategy to control the hydrodynamic force on fluidic pinball, i.e., force extremum and tracking, from cylinders' rotation. By adequately designing reward functions and encoding historical observations, and after automatic learning of thousands of iterations, the DRL-based control was shown to make reasonable and valid control decisions in nonparametric control parameter space, which is comparable to and even better than the optimal policy found through lengthy brute-force searching. Subsequently, one of these results was analyzed by a machine learning model that enabled us to shed light on the basis of decision-making and physical mechanisms of the force tracking process. The finding from this work can control hydrodynamic force on the operation of fluidic pinball system and potentially pave the way for exploring efficient active flow control strategies in other complex fluid dynamic problems.

References in corpus (25)

• Machine Learning for Fluid Mechanics
• Addressing Function Approximation Error in Actor-Critic Methods
• Artificial Neural Networks trained through Deep Reinforcement Learning discover control strategies for active flow control
• Data-driven prediction of unsteady flow fields over a circular cylinder using deep learning
• Applying deep reinforcement learning to active flow control in turbulent conditions
• Verifiable Reinforcement Learning via Policy Extraction
• Explainable Deep Reinforcement Learning: State of the Art and Challenges
• From bypass transition to flow control and data-driven turbulence modeling: An input-output viewpoint
• A review on deep reinforcement learning for fluid mechanics: an update
• A review on Deep Reinforcement Learning for Fluid Mechanics
• Reinforcement-learning-based control of confined cylinder wakes with stability analyses
• Data-Driven Model Predictive Control using Interpolated Koopman Generators
• Low-order model for successive bifurcations of the fluidic pinball
• DRLinFluids -- An open-source python platform of coupling Deep Reinforcement Learning and OpenFOAM
• Comparative analysis of machine learning methods for active flow control
• Bluff body uses deep-reinforcement-learning trained active flow control to achieve hydrodynamic stealth
• Stabilization of the fluidic pinball with gradient-enriched machine learning control
• Network broadcast analysis and control of turbulent flows
• Reinforcement-learning-based control of convectively-unstable flows
• Metric for attractor overlap
• Machine-learned control-oriented flow estimation for multiactuator multi-sensor systems exemplified for the fluidic pinball
• Linear feedback control of invariant solutions in channel flow
• Reduced-order modeling of the fluidic pinball
• xMLC -- A Toolkit for Machine Learning Control
• Deep reinforcement learning for flow control exploits different physics for increasing Reynolds-number regimes