Reinforcement learning for pursuit and evasion of microswimmers at low Reynolds number

Abstract

We consider a model of two competing microswimming agents engaged in a pursue-evasion task within a low-Reynolds-number environment. Agents can only perform simple maneuvers and sense hydrodynamic disturbances, which provide ambiguous (partial) information about the opponent's position and motion. We frame the problem as a zero-sum game: The pursuer has to capture the evader in the shortest time, while the evader aims at deferring capture as long as possible. We show that the agents, trained via adversarial reinforcement learning, are able to overcome partial observability by discovering increasingly complex sequences of moves and countermoves that outperform known heuristic strategies and exploit the hydrodynamic environment.

20 pages, 5 figures (Supplementary Material in ancillary directory)

References in corpus (3)

• A Brief Survey of Deep Reinforcement Learning
• The hydrodynamics of swimming microorganisms
• Active gyrotactic stability of microswimmers using hydromechanical signals

Cited by in corpus (11)

• Optimal active particle navigation meets machine learning
• Optimising low-Reynolds-number predation via optimal control and reinforcement learning
• Jeffery's orbits and microswimmers in flows: A theoretical review
• Noisy Pursuit and Pattern Formation of Self-Steering Active Particles
• Long-distance migration with minimal energy consumption in a thermal turbulent environment
• Hydrodynamic pursuit by cognitive self-steering microswimmers
• Taming Lagrangian Chaos with Multi-Objective Reinforcement Learning
• Learning to swim efficiently in a nonuniform flow field
• Super-resolution reconstruction of turbulent flows from a single Lagrangian trajectory
• Learning to traverse convective flows at moderate to high Rayleigh numbers
• Run-and-Tumble Escape in Pursuit-Evasion Dynamics of Intelligent Active Particles