Self-learning how to swim at low Reynolds number

Abstract

Synthetic microswimmers show great promise in biomedical applications such as drug delivery and microsurgery. Their locomotion, however, is subject to stringent constraints due to the dominance of viscous over inertial forces at low Reynolds number (Re) in the microscopic world. Furthermore, locomotory gaits designed for one medium may become ineffective in a different medium. Successful biomedical applications of synthetic microswimmers rely on their ability to traverse biological environments with vastly different properties. Here we leverage the prowess of machine learning to present an alternative approach to designing low Re swimmers. Instead of specifying any locomotory gaits \textit{a priori}, here a swimmer develops its own propulsion strategy based on its interactions with the surrounding medium via reinforcement learning. This self-learning capability enables the swimmer to modify its propulsion strategy in response to different environments. We illustrate this new approach using a minimal example that integrates a standard reinforcement learning algorithm ($Q$-learning) into the locomotion of a swimmer consisting of an assembly of spheres connected by extensible rods. We showcase theoretically that this first self-learning swimmer can recover a previously known propulsion strategy without prior knowledge in low Re locomotion, identify more effective locomotory gaits when the number of spheres increases, and adapt its locomotory gaits in different media. These results represent initial steps towards the design of a new class of self-learning, adaptive (or "smart") swimmers with robust locomotive capabilities to traverse complex biological environments.

References in corpus (16)

• The hydrodynamics of swimming microorganisms
• Physics of Microswimmers - Single Particle Motion and Collective Behavior
• A Simplest Swimmer at Low Reynolds Number: Three Linked Spheres
• Bacterial hydrodynamics
• Adaptive locomotion of artificial microswimmers
• Flow Navigation by Smart Microswimmers via Reinforcement Learning
• High-Speed Propulsion of Flexible Nanowire Motors: Theory and Experiments
• Pushmepullyou: An efficient micro-swimmer
• A basic swimmer at low Reynolds number
• Learning to school in the presence of hydrodynamic interactions
• Classifying vortex wakes using neural networks
• Geometric visualization of self-propulsion in a complex medium
• Realization of the Najafi-Golestanian microswimmer
• Optimally swimming Stokesian robots
• Stochastic Low Reynolds Number Swimmers
• Efficiency limits of the three-sphere swimmer

Cited by in corpus (23)

• Microswimmers learning chemotaxis with genetic algorithms
• Hydrodynamics can Determine the Optimal Route for Microswimmer Navigation
• Learning to swim in potential flow
• Navigation of micro-swimmers in steady flow: the importance of symmetries
• Independent Control and Path Planning of Microswimmers with a Uniform Magnetic Field
• Optimal active particle navigation meets machine learning
• Optimising low-Reynolds-number predation via optimal control and reinforcement learning
• Jeffery's orbits and microswimmers in flows: A theoretical review
• Active Cloaking in Stokes Flows via Reinforcement Learning
• Hunting active Brownian particles: Learning optimal behavior
• Long-distance migration with minimal energy consumption in a thermal turbulent environment
• Generalized Three-Sphere Microswimmers
• Learning how to find targets in the micro-world: The case of intermittent active Brownian particles
• Neuroevolution of Decentralized Decision-Making in N-Bead Swimmers Leads to Scalable and Robust Collective Locomotion
• Stabilising viscous extensional flows using Reinforcement Learning
• Chemoreception and chemotaxis of a three-sphere swimmer
• The effect of axisymmetric confinement on propulsion of a three-sphere microswimmer
• Adaptive active Brownian particles searching for targets of unknown positions
• Reinforcement Learning for Active Matter
• Propulsion of a three-sphere micro-robot in a porous medium
• Inferring activity from the flow field around active colloidal particles using deep learning
• Run-and-Tumble Particles Learning Chemotaxis
• Reinforcement learning of a biflagellate model microswimmer