Controlling Rayleigh-Bénard convection via Reinforcement Learning

Abstract

Thermal convection is ubiquitous in nature as well as in many industrial applications. The identification of effective control strategies to, e.g., suppress or enhance the convective heat exchange under fixed external thermal gradients is an outstanding fundamental and technological issue. In this work, we explore a novel approach, based on a state-of-the-art Reinforcement Learning (RL) algorithm, which is capable of significantly reducing the heat transport in a two-dimensional Rayleigh-Bénard system by applying small temperature fluctuations to the lower boundary of the system. By using numerical simulations, we show that our RL-based control is able to stabilize the conductive regime and bring the onset of convection up to a Rayleigh number $R{a}_c\approx 3\cdot 10^4$, whereas in the uncontrolled case it holds $R{a}_c=1708$. Additionally, for $Ra>3\cdot 10^4$, our approach outperforms other state-of-the-art control algorithms reducing the heat flux by a factor of about $2.5$. In the last part of the manuscript, we address theoretical limits connected to controlling an unstable and chaotic dynamics as the one considered here. We show that controllability is hindered by observability and/or capabilities of actuating actions, which can be quantified in terms of characteristic time delays. When these delays become comparable with the Lyapunov time of the system, control becomes impossible.

arXiv comment24 pages, 10 figures

References in corpus (11)

• Machine Learning for Fluid Mechanics
• Machine learning and the physical sciences
• Artificial Neural Networks trained through Deep Reinforcement Learning discover control strategies for active flow control
• Flow Navigation by Smart Microswimmers via Reinforcement Learning
• Accelerating Deep Reinforcement Learning strategies of Flow Control through a multi-environment approach
• Learning to school in the presence of hydrodynamic interactions
• Zermelo's problem: Optimal point-to-point navigation in 2D turbulent flows using Reinforcement Learning
• Big Data: the End of the Scientific Method?
• Machine learning strategies for path-planning microswimmers in turbulent flows
• Finding Efficient Swimming Strategies in a Three Dimensional Chaotic Flow by Reinforcement Learning
• Smart Inertial Particles

Cited by in corpus (27)

• Deep Neural Networks for Nonlinear Model Order Reduction of Unsteady Flows
• Applying deep reinforcement learning to active flow control in turbulent conditions
• Recent advances in applying deep reinforcement learning for flow control: perspectives and future directions
• A review on Deep Reinforcement Learning for Fluid Mechanics
• A review on deep reinforcement learning for fluid mechanics: an update
• Reinforcement-learning-based control of confined cylinder wakes with stability analyses
• Reinforcement Learning of Control Strategies for Reducing Skin Friction Drag in a Fully Developed Channel Flow
• DRLinFluids -- An open-source python platform of coupling Deep Reinforcement Learning and OpenFOAM
• Comparative analysis of machine learning methods for active flow control
• Effective control of two-dimensional Rayleigh--Bénard convection: invariant multi-agent reinforcement learning is all you need
• Deep reinforcement learning for large-eddy simulation modeling in wall-bounded turbulence
• Machine learning flow control with few sensor feedback and measurement noise
• Multi-fidelity reinforcement learning framework for shape optimization
• Reinforcement-learning-based actuator selection method for active flow control
• Data-driven control of spatiotemporal chaos with reduced-order neural ODE-based models and reinforcement learning
• Reinforcement-learning-based control of convectively-unstable flows
• Discovering explicit Reynolds-averaged turbulence closures for turbulent separated flows through deep learning-based symbolic regression with non-linear corrections
• Towards learning Lattice Boltzmann collision operators
• Genetically-inspired convective heat transfer enhancement in a turbulent boundary layer
• Generalizability of reservoir computing for flux-driven two-dimensional convection
• Active flow control of a turbulent separation bubble through deep reinforcement learning
• SINDy-RL: Interpretable and Efficient Model-Based Reinforcement Learning
• Self-tuning model predictive control for wake flows
• Deep reinforcement learning for the management of the wall regeneration cycle in wall-bounded turbulent flows
• Using machine-learning modelling to understand macroscopic dynamics in a system of coupled maps
• Physics-informed Dyna-Style Model-Based Deep Reinforcement Learning for Dynamic Control
• Large-scale convective flow sustained by thermally active Lagrangian tracers