How to Control Hydrodynamic Force on Fluidic Pinball via Deep Reinforcement Learning

Abstract

Deep reinforcement learning (DRL) for fluidic pinball, three individually rotating cylinders in the uniform flow arranged in an equilaterally triangular configuration, can learn the efficient flow control strategies due to the validity of self-learning and data-driven state estimation for complex fluid dynamic problems. In this work, we present a DRL-based real-time feedback strategy to control the hydrodynamic force on fluidic pinball, i.e., force extremum and tracking, from cylinders' rotation. By adequately designing reward functions and encoding historical observations, and after automatic learning of thousands of iterations, the DRL-based control was shown to make reasonable and valid control decisions in nonparametric control parameter space, which is comparable to and even better than the optimal policy found through lengthy brute-force searching. Subsequently, one of these results was analyzed by a machine learning model that enabled us to shed light on the basis of decision-making and physical mechanisms of the force tracking process. The finding from this work can control hydrodynamic force on the operation of fluidic pinball system and potentially pave the way for exploring efficient active flow control strategies in other complex fluid dynamic problems.

References in corpus (16)

• Applying deep reinforcement learning to active flow control in turbulent conditions
• Explainable Deep Reinforcement Learning: State of the Art and Challenges
• From bypass transition to flow control and data-driven turbulence modeling: An input-output viewpoint
• A review on deep reinforcement learning for fluid mechanics: an update
• Reinforcement-learning-based control of confined cylinder wakes with stability analyses
• Data-Driven Model Predictive Control using Interpolated Koopman Generators
• DRLinFluids -- An open-source python platform of coupling Deep Reinforcement Learning and OpenFOAM
• Comparative analysis of machine learning methods for active flow control
• Bluff body uses deep-reinforcement-learning trained active flow control to achieve hydrodynamic stealth
• Stabilization of the fluidic pinball with gradient-enriched machine learning control
• Network broadcast analysis and control of turbulent flows
• Reinforcement-learning-based control of convectively-unstable flows
• Machine-learned control-oriented flow estimation for multiactuator multi-sensor systems exemplified for the fluidic pinball
• Reduced-order modeling of the fluidic pinball
• Deep reinforcement learning for flow control exploits different physics for increasing Reynolds-number regimes
• xMLC -- A Toolkit for Machine Learning Control