Long-distance migration with minimal energy consumption in a thermal turbulent environment

Abstract

We adopt the reinforcement learning algorithm to train the self-propelling agent migrating long-distance in a thermal turbulent environment. We choose the Rayleigh-Bénard turbulent convection cell with an aspect ratio ($\Gamma$, which is defined as the ratio between cell length and cell height) of 2 as the training environment. Our results showed that, compared to a naive agent that moves straight from the origin to the destination, the smart agent can learn to utilize the carrier flow currents to save propelling energy. We then apply the optimal policy obtained from the $\Gamma =2$ cell and test the smart agent migrating in convection cells with $\Gamma$ up to 32. In a larger $\Gamma$ cell, the dominant flow modes of horizontally stacked rolls are less stable, and the energy contained in higher-order flow modes increases. We found that the optimized policy can be successfully extended to convection cells with a larger $\Gamma$. In addition, the ratio of propelling energy consumed by the smart agent to that of the naive agent decreases with the increase of $\Gamma$, indicating more propelling energy can be saved by the smart agent in a larger $\Gamma$ cell. We also evaluate the optimized policy when the agents are being released from the randomly chosen origin, which aims to test the robustness of the learning framework, and possible solutions to improve the success rate are suggested. This work has implications for long-distance migration problems, such as unmanned aerial vehicles patrolling in a turbulent convective environment, where planning energy-efficient trajectories can be beneficial to increase their endurance.

32 pages, 14 figures

References in corpus (10)

• Multiple states in turbulent large-aspect ratio thermal convection: What determines the number of convection rolls?
• Thermal soaring flight of birds and unmanned aerial vehicles
• Multi-GPU thermal lattice Boltzmann simulations using OpenACC and MPI
• Comparing bird and human soaring strategies
• Navigation of micro-swimmers in steady flow: the importance of symmetries
• Reinforcement learning for pursuit and evasion of microswimmers at low Reynolds number
• Turbulence explains the accelerations of an eagle in natural flight
• Finite-Horizon, Energy-Optimal Trajectories in Unsteady Flows
• Production and transport of vorticity in two-dimensional Rayleigh-Bénard convection cell
• Migration of self-propelling agent in a turbulent environment with minimal energy consumption

Cited by in corpus (5)

• Optimal tracking strategies in a turbulent flow
• Short term vs. long term: optimization of microswimmer navigation on different time horizons
• Efficient survival strategy for zooplankton in turbulence
• Super-resolution reconstruction of turbulent flows from a single Lagrangian trajectory
• Learning to traverse convective flows at moderate to high Rayleigh numbers