Effective control of two-dimensional Rayleigh--Bénard convection: invariant multi-agent reinforcement learning is all you need

Abstract

Rayleigh-Bénard convection (RBC) is a recurrent phenomenon in several industrial and geoscience flows and a well-studied system from a fundamental fluid-mechanics viewpoint. However, controlling RBC, for example by modulating the spatial distribution of the bottom-plate heating in the canonical RBC configuration, remains a challenging topic for classical control-theory methods. In the present work, we apply deep reinforcement learning (DRL) for controlling RBC. We show that effective RBC control can be obtained by leveraging invariant multi-agent reinforcement learning (MARL), which takes advantage of the locality and translational invariance inherent to RBC flows inside wide channels. The MARL framework applied to RBC allows for an increase in the number of control segments without encountering the curse of dimensionality that would result from a naive increase in the DRL action-size dimension. This is made possible by the MARL ability for re-using the knowledge generated in different parts of the RBC domain. We show in a case study that MARL DRL is able to discover an advanced control strategy that destabilizes the spontaneous RBC double-cell pattern, changes the topology of RBC by coalescing adjacent convection cells, and actively controls the resulting coalesced cell to bring it to a new stable configuration. This modified flow configuration results in reduced convective heat transfer, which is beneficial in several industrial processes. Therefore, our work both shows the potential of MARL DRL for controlling large RBC systems, as well as demonstrates the possibility for DRL to discover strategies that move the RBC configuration between different topological configurations, yielding desirable heat-transfer characteristics. These results are useful for both gaining further understanding of the intrinsic properties of RBC, as well as for developing industrial applications.

34 pages, 11 figures submitted to Physics of Fluids

References in corpus (17)

• Closed-loop separation control using machine learning
• Robust active flow control over a range of Reynolds numbers using an artificial neural network trained through deep reinforcement learning
• Applying deep reinforcement learning to active flow control in turbulent conditions
• Recent advances in applying deep reinforcement learning for flow control: perspectives and future directions
• Tianshou: a Highly Modularized Deep Reinforcement Learning Library
• Reinforcement-learning-based control of confined cylinder wakes with stability analyses
• Artificial intelligence control of a turbulent jet
• Controlling Rayleigh-Bénard convection via Reinforcement Learning
• Reinforcement Learning of Control Strategies for Reducing Skin Friction Drag in a Fully Developed Channel Flow
• DRLinFluids -- An open-source python platform of coupling Deep Reinforcement Learning and OpenFOAM
• Comparative analysis of machine learning methods for active flow control
• Bluff body uses deep-reinforcement-learning trained active flow control to achieve hydrodynamic stealth
• Deep Reinforcement Learning for Computational Fluid Dynamics on HPC Systems
• Stabilization of the fluidic pinball with gradient-enriched machine learning control
• Reinforcement-learning-based actuator selection method for active flow control
• Reinforcement-learning-based control of convectively-unstable flows
• An application of data driven reward of deep reinforcement learning by dynamic mode decomposition in active flow control

Cited by in corpus (5)

• Latent feedback control of distributed systems in multiple scenarios through deep learning-based reduced order models
• Real-time optimal control of high-dimensional parametrized systems by deep learning-based reduced order models
• Optimal mesh generation for a non-iterative grid-converged solution of flow through a blade passage using deep reinforcement learning
• Deep reinforcement learning for the management of the wall regeneration cycle in wall-bounded turbulent flows
• Reinforcement-learning-assisted control of four-roll mills: geometric symmetry and inertial effect