Finding Efficient Swimming Strategies in a Three Dimensional Chaotic Flow by Reinforcement Learning

Abstract

We apply a reinforcement learning algorithm to show how smart particles can learn approximately optimal strategies to navigate in complex flows. In this paper we consider microswimmers in a paradigmatic three-dimensional case given by a stationary superposition of two Arnold-Beltrami-Childress flows with chaotic advection along streamlines. In such a flow, we study the evolution of point-like particles which can decide in which direction to swim, while keeping the velocity amplitude constant. We show that it is sufficient to endow the swimmers with a very restricted set of actions (six fixed swimming directions in our case) to have enough freedom to find efficient strategies to move upward and escape local fluid traps. The key ingredient is the learning-from-experience structure of the algorithm, which assigns positive or negative rewards depending on whether the taken action is, or is not, profitable for the predetermined goal in the long term horizon. This is another example supporting the efficiency of the reinforcement learning approach to learn how to accomplish difficult tasks in complex fluid environments.

Published on Eur. Phys. J. E (December 14, 2017)

References in corpus (5)

• The hydrodynamics of swimming microorganisms
• Flow Navigation by Smart Microswimmers via Reinforcement Learning
• Nonlinear response of inertial tracers in steady laminar flows: differential and absolute negative mobility
• Gyrotactic trapping in laminar and turbulent Kolmogorov flow
• Anomalous force-velocity relation of driven inertial tracers in steady laminar flows

Cited by in corpus (30)

• Zermelo's problem: Optimal point-to-point navigation in 2D turbulent flows using Reinforcement Learning
• Machine learning strategies for path-planning microswimmers in turbulent flows
• Controlling Rayleigh-Bénard convection via Reinforcement Learning
• Optimal steering of a smart active particle
• Microswimmers learning chemotaxis with genetic algorithms
• Hydrodynamics can Determine the Optimal Route for Microswimmer Navigation
• Searching for a source without gradients: how good is infotaxis and how to beat it
• Learning to swim in potential flow
• Smart Inertial Particles
• Reinforcement learning of optimal active particle navigation
• Navigation of micro-swimmers in steady flow: the importance of symmetries
• Surfing on turbulence: a strategy for planktonic navigation
• Optimal active particle navigation meets machine learning
• Jeffery's orbits and microswimmers in flows: A theoretical review
• Swimming strategy of settling elongated microswimmers by reinforcement learning
• Hydrodynamic effects on the liquid-hexatic transition of active colloids
• Long-distance migration with minimal energy consumption in a thermal turbulent environment
• Reinforcement Learning of Artificial Microswimmers
• Learning how to find targets in the micro-world: The case of intermittent active Brownian particles
• Learning to swim efficiently in a nonuniform flow field
• Short term vs. long term: optimization of microswimmer navigation on different time horizons
• Migration of self-propelling agent in a turbulent environment with minimal energy consumption
• Stabilising viscous extensional flows using Reinforcement Learning
• A critical assessment of reinforcement learning methods for microswimmer navigation in complex flows
• Adaptive active Brownian particles searching for targets of unknown positions
• Active gyrotactic stability of microswimmers using hydromechanical signals
• Run-and-Tumble Particles Learning Chemotaxis
• Physics-informed Dyna-Style Model-Based Deep Reinforcement Learning for Dynamic Control
• Optimal navigation in two-dimensional flows: Control theory and reinforcement learning
• Learning to traverse convective flows at moderate to high Rayleigh numbers