Flow Navigation by Smart Microswimmers via Reinforcement Learning
arXiv:1701.08848 · doi:10.1103/PhysRevLett.118.158004
Abstract
Smart active particles can acquire some limited knowledge of the fluid environment from simple mechanical cues and exert a control on their preferred steering direction. Their goal is to learn the best way to navigate by exploiting the underlying flow whenever possible. As an example, we focus our attention on smart gravitactic swimmers. These are active particles whose task is to reach the highest altitude within some time horizon, given the constraints enforced by fluid mechanics. By means of numerical experiments, we show that swimmers indeed learn nearly optimal strategies just by experience. A reinforcement learning algorithm allows particles to learn effective strategies even in difficult situations when, in the absence of control, they would end up being trapped by flow structures. These strategies are highly nontrivial and cannot be easily guessed in advance. This Letter illustrates the potential of reinforcement learning algorithms to model adaptive behavior in complex flows and paves the way towards the engineering of smart microswimmers that solve difficult navigation problems.
Published on Physical Review Letters (April 12, 2017)
References in corpus (4)
Cited by in corpus (91)
- Machine Learning for Fluid Mechanics
- A high-bias, low-variance introduction to Machine Learning for physicists
- The 2019 Motile Active Matter Roadmap
- Reinforcement Learning in Different Phases of Quantum Control
- Flow Navigation by Smart Microswimmers via Reinforcement Learning
- Robust active flow control over a range of Reynolds numbers using an artificial neural network trained through deep reinforcement learning
- When does reinforcement learning stand out in quantum control? A comparative study on state preparation
- Zermelo's problem: Optimal point-to-point navigation in 2D turbulent flows using Reinforcement Learning
- Learning Efficient Navigation in Vortical Flow Fields
- Machine learning with neural networks
- Machine learning strategies for path-planning microswimmers in turbulent flows
- Controlling Rayleigh-Bénard convection via Reinforcement Learning
- Coherent Transport of Quantum States by Deep Reinforcement Learning
- Optimal steering of a smart active particle
- Finding Efficient Swimming Strategies in a Three Dimensional Chaotic Flow by Reinforcement Learning
- Reinforcement learning for autonomous preparation of Floquet-engineered states: Inverting the quantum Kapitza oscillator
- Self-learning how to swim at low Reynolds number
- Microswimmers learning chemotaxis with genetic algorithms
- Hydrodynamics can Determine the Optimal Route for Microswimmer Navigation
- Neural Network Models for the Anisotropic Reynolds Stress Tensor in Turbulent Channel Flow
- Bluff body uses deep-reinforcement-learning trained active flow control to achieve hydrodynamic stealth
- Deep reinforcement learning for large-eddy simulation modeling in wall-bounded turbulence
- Programmable Control of Ultrasound Swarmbots through Reinforcement Learning
- Searching for a source without gradients: how good is infotaxis and how to beat it
- Learning to swim in potential flow
- Smart Inertial Particles
- Learning to Control Active Matter
- Reinforcement learning of optimal active particle navigation
- Navigation of micro-swimmers in steady flow: the importance of symmetries
- Reinforcement learning for pursuit and evasion of microswimmers at low Reynolds number
- Surfing on turbulence: a strategy for planktonic navigation
- Independent Control and Path Planning of Microswimmers with a Uniform Magnetic Field
- Optimal active particle navigation meets machine learning
- Optimising low-Reynolds-number predation via optimal control and reinforcement learning
- Photonic architecture for reinforcement learning
- Jeffery's orbits and microswimmers in flows: A theoretical review
- Designing, Synthesizing and Modeling Active Fluids
- Active Cloaking in Stokes Flows via Reinforcement Learning
- Discovering explicit Reynolds-averaged turbulence closures for turbulent separated flows through deep learning-based symbolic regression with non-linear corrections
- Swimming strategy of settling elongated microswimmers by reinforcement learning
- Enhancing Efficiency and Propulsion in Bio-mimetic Robotic Fish through End-to-End Deep Reinforcement Learning
- Noisy Pursuit and Pattern Formation of Self-Steering Active Particles
- Optimal navigation of microswimmers in complex and noisy environments
- Optimal navigation strategies for microswimmers on curved manifolds
- Anomalous force-velocity relation of driven inertial tracers in steady laminar flows
- Anomalous mobility of a driven active particle in a steady laminar flow
- Hunting active Brownian particles: Learning optimal behavior
- Reinforcement Learning of Artificial Microswimmers
- Remember and Forget for Experience Replay
- Long-distance migration with minimal energy consumption in a thermal turbulent environment
- Energetic cost of microswimmer navigation: the role of body shape
- Learning how to find targets in the micro-world: The case of intermittent active Brownian particles
- Taming Polar Active Matter with Moving Substrates: Directed Transport and Counterpropagating Macrobands
- The statistical properties of turbulence in the presence of a smart small-scale control
- Efficient collective swimming by harnessing vortices through deep reinforcement learning
- Reinforcement Learning for Active Flow Control in Experiments
- High-dimensional reinforcement learning for optimization and control of ultracold quantum gases
- Noise-Robust End-to-End Quantum Control using Deep Autoregressive Policy Networks
- The control of particles in the Stokes limit
- SINDy-RL: Interpretable and Efficient Model-Based Reinforcement Learning
- Optimal Navigation in Microfluidics via the Optimization of a Discrete Loss
- Learning optimal integration of spatial and temporal information in noisy chemotaxis
- Learning to swim efficiently in a nonuniform flow field
- Microswimmer trapping in surface waves with shear
- Coordinated Crawling via Reinforcement Learning
- Thermodynamic efficiency of learning a rule in neural networks
- Short term vs. long term: optimization of microswimmer navigation on different time horizons
- Optimal Control in Soft and Active Matter
- Neuroevolution of Decentralized Decision-Making in N-Bead Swimmers Leads to Scalable and Robust Collective Locomotion
- Migration of self-propelling agent in a turbulent environment with minimal energy consumption
- Stabilising viscous extensional flows using Reinforcement Learning
- Efficient survival strategy for zooplankton in turbulence
- Adaptive active Brownian particles searching for targets of unknown positions
- A critical assessment of reinforcement learning methods for microswimmer navigation in complex flows
- Reinforcement Learning for Active Matter
- Reinforcement Learning reveals fundamental limits on the mixing of active particles
- Kinetic theory of decentralized learning for smart active matter
- Policy heterogeneity improves collective olfactory search in 3-D turbulence
- Quantum Compiling by Deep Reinforcement Learning
- Run-and-Tumble Particles Learning Chemotaxis
- Active gyrotactic stability of microswimmers using hydromechanical signals
- Gradient dynamics in reinforcement learning
- Leveraging three-dimensionality for navigation in bluff-body wakes
- Smart navigation of a gravity-driven glider with adjustable centre-of-mass
- Deep-Reinforcement-Learning for Gliding and Perching Bodies
- Learning swimming escape patterns for larval fish under energy constraints
- Optimizing collective fieldtaxis of swarming agents through reinforcement learning
- Optimal navigation in two-dimensional flows: Control theory and reinforcement learning
- Symmetry reduction for deep reinforcement learning active control of chaotic spatiotemporal dynamics
- A study of first-passage time minimization via Q-learning in heated gridworlds
- Learning to traverse convective flows at moderate to high Rayleigh numbers