Migration of self-propelling agent in a turbulent environment with minimal energy consumption

Abstract

We present a numerical study of training a self-propelling agent to migrate in the unsteady flow environment. We control the agent to utilize the background flow structure by adopting the reinforcement learning algorithm to minimize energy consumption. We considered the agent migrating in two types of flows: one is simple periodical double-gyre flow as a proof-of-concept example, while the other is complex turbulent Rayleigh-Bénard convection as a paradigm for migrating in the convective atmosphere or the ocean. The results show that the smart agent in both flows can learn to migrate from one position to another while utilizing background flow currents as much as possible to minimize the energy consumption, which is evident by comparing the smart agent with a naive agent that moves straight from the origin to the destination. In addition, we found that compared to the double-gyre flow, the flow field in the turbulent Rayleigh-Bénard convection exhibits more substantial fluctuations, and the training agent is more likely to explore different migration strategies; thus, the training process is more difficult to converge. Nevertheless, we can still identify an energy-efficient trajectory that corresponds to the strategy with the highest reward received by the agent. These results have important implications for many migration problems such as unmanned aerial vehicles flying in a turbulent convective environment, where planning energy-efficient trajectories are often involved.

arXiv comment23 pages, 12 figures

References in corpus (17)

• Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor
• Machine Learning for Fluid Mechanics
• Machine learning and the physical sciences
• A high-bias, low-variance introduction to Machine Learning for physicists
• Flow Navigation by Smart Microswimmers via Reinforcement Learning
• Zermelo's problem: Optimal point-to-point navigation in 2D turbulent flows using Reinforcement Learning
• Lattice Boltzmann simulations of three-dimensional thermal convective flows at high Rayleigh number
• Machine learning strategies for path-planning microswimmers in turbulent flows
• Thermal soaring flight of birds and unmanned aerial vehicles
• Finding Efficient Swimming Strategies in a Three Dimensional Chaotic Flow by Reinforcement Learning
• Comparing bird and human soaring strategies
• Correlation of internal flow structure with heat transfer efficiency in turbulent Rayleigh-Bénard convection
• Smart Inertial Particles
• Navigation of micro-swimmers in steady flow: the importance of symmetries
• Turbulence explains the accelerations of an eagle in natural flight
• Swimming strategy of settling elongated microswimmers by reinforcement learning
• Production and transport of vorticity in two-dimensional Rayleigh-Bénard convection cell

Cited by in corpus (3)

• Long-distance migration with minimal energy consumption in a thermal turbulent environment
• Super-resolution reconstruction of turbulent flows from a single Lagrangian trajectory
• Learning to traverse convective flows at moderate to high Rayleigh numbers