Learning to swim in potential flow

Abstract

Fish swim by undulating their bodies. These propulsive motions require coordinated shape changes of a body that interacts with its fluid environment, but the specific shape coordination that leads to robust turning and swimming motions remains unclear. To address the problem of underwater motion planning, we propose a simple model of a three-link fish swimming in a potential flow environment and we use model-free reinforcement learning for shape control. We arrive at optimal shape changes for two swimming tasks: swimming in a desired direction and swimming towards a known target. This fish model belongs to a class of problems in geometric mechanics, known as driftless dynamical systems, which allow us to analyze the swimming behavior in terms of geometric phases over the shape space of the fish. These geometric methods are less intuitive in the presence of drift. Here, we use the shape space analysis as a tool for assessing, visualizing, and interpreting the control policies obtained via reinforcement learning in the absence of drift. We then examine the robustness of these policies to drift-related perturbations. Although the fish has no direct control over the drift itself, it learns to take advantage of the presence of moderate drift to reach its target.

References in corpus (10)

• Adam: A Method for Stochastic Optimization
• Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor
• Flow Navigation by Smart Microswimmers via Reinforcement Learning
• Model of collective fish behavior with hydrodynamic interactions
• Learning to school in the presence of hydrodynamic interactions
• Classifying vortex wakes using neural networks
• Finding Efficient Swimming Strategies in a Three Dimensional Chaotic Flow by Reinforcement Learning
• Self-learning how to swim at low Reynolds number
• Fishlike Rheotaxis
• Optimal sensing for fish school identification

Cited by in corpus (8)

• Recent advances in applying deep reinforcement learning for flow control: perspectives and future directions
• Learning Efficient Navigation in Vortical Flow Fields
• Optimising low-Reynolds-number predation via optimal control and reinforcement learning
• Quantum support vector machines for aerodynamic classification
• Evaluating Evasion Strategies in Zebrafish Larvae
• Stabilising viscous extensional flows using Reinforcement Learning
• Leveraging three-dimensionality for navigation in bluff-body wakes
• Optimum control strategies for maximum thrust production in underwater undulatory swimming