Learning to traverse convective flows at moderate to high Rayleigh numbers

Abstract

We study the navigation of a self-propelled inertial particle in two-dimensional Rayleigh-Bénard convection at Prandtl number $Pr=0.71$ and cell aspect ratio $\Gamma =4$ for Rayleigh numbers $Ra$ ranging from $10^7$ to $10^{11}$. A reinforcement-learning (RL) controller selects the propulsive acceleration, subject to an upper bound ${\mathcal{A}}_{\max }$, to achieve a prescribed horizontal displacement. We find that the success rate increases abruptly with ${\mathcal{A}}_{\max }$ at moderate $Ra$, whereas at higher $Ra$ the transition becomes more gradual and shifts to larger ${\mathcal{A}}_{\max }$. Moreover, although the completion time increases with $Ra$, the propulsion energy required for successful traversal decreases. Proper orthogonal decomposition indicates that these performance differences are associated with reorganisation of the carrier flow. At moderate $Ra$, the dominant large-scale circulation partitions the domain through persistent transport barriers, requiring a finite thrust surplus to cross them; at higher $Ra$, energy is distributed across many modes, the barriers fragment and transient plume-assisted pathways emerge. Compared with a constant-heading baseline, the learned policy aligns with local currents and consumes significantly less energy. Lagrangian coherent structure analysis further suggests that the RL agent tends to cross repelling barriers and surf along attracting pathways. Finally, by mapping these behaviours onto the local Eulerian flow topology using Voronoi tessellation and the $Q$-criterion, we distil an interpretable, physics-based heuristic strategy that retains robust navigability. These results connect turbulent-flow organisation with autonomous navigation under bounded actuation.

arXiv comment37 pages, 21 figures

References in corpus (29)

• Active Particles in Complex and Crowded Environments
• Machine Learning for Fluid Mechanics
• The hydrodynamics of swimming microorganisms
• A high-bias, low-variance introduction to Machine Learning for physicists
• Flow Navigation by Smart Microswimmers via Reinforcement Learning
• Turbulent superstructures in Rayleigh-Bénard convection
• Transition to the ultimate regime in two-dimensional Rayleigh-Bénard convection
• Zermelo's problem: Optimal point-to-point navigation in 2D turbulent flows using Reinforcement Learning
• Comparison of computational codes for direct numerical simulations of turbulent Rayleigh-Bénard convection
• Multiple states in turbulent large-aspect ratio thermal convection: What determines the number of convection rolls?
• Optimal steering of a smart active particle
• Finding Efficient Swimming Strategies in a Three Dimensional Chaotic Flow by Reinforcement Learning
• Preferential sampling and small-scale clustering of gyrotactic microswimmers in turbulence
• Comparing bird and human soaring strategies
• Searching for a source without gradients: how good is infotaxis and how to beat it
• Reinforcement learning of optimal active particle navigation
• Navigation of micro-swimmers in steady flow: the importance of symmetries
• Reinforcement learning for pursuit and evasion of microswimmers at low Reynolds number
• Surfing on turbulence: a strategy for planktonic navigation
• Turbulence explains the accelerations of an eagle in natural flight
• Optimising low-Reynolds-number predation via optimal control and reinforcement learning
• Finite-Horizon, Energy-Optimal Trajectories in Unsteady Flows
• Large-eddy simulation of Rayleigh-Bénard convection at extreme Rayleigh numbers
• Lagrangian tracer dynamics in a closed cylindrical turbulent convection cell
• Long-distance migration with minimal energy consumption in a thermal turbulent environment
• Energetic cost of microswimmer navigation: the role of body shape
• Lagrangian coherent sets in turbulent Rayleigh-Bénard convection
• Many wrong models approach to localize an odor source in turbulence with static sensors
• Migration of self-propelling agent in a turbulent environment with minimal energy consumption