Deep reinforcement learning for tracking a moving target in jellyfish-like swimming

Abstract

We develop a deep reinforcement learning method for training a jellyfish-like swimmer to effectively track a moving target in a two-dimensional flow. This swimmer is a flexible object equipped with a muscle model based on torsional springs. We employ a deep Q-network (DQN) that takes the swimmer's geometry and dynamic parameters as inputs, and outputs actions which are the forces applied to the swimmer. In particular, we introduce an action regulation to mitigate the interference from complex fluid-structure interactions. The goal of these actions is to navigate the swimmer to a target point in the shortest possible time. In the DQN training, the data on the swimmer's motions are obtained from simulations conducted using the immersed boundary method. During tracking a moving target, there is an inherent delay between the application of forces and the corresponding response of the swimmer's body due to hydrodynamic interactions between the shedding vortices and the swimmer's own locomotion. Our tests demonstrate that the swimmer, with the DQN agent and action regulation, is able to dynamically adjust its course based on its instantaneous state. This work extends the application scope of machine learning in controlling flexible objects within fluid environments.

22pages,14 figures

References in corpus (12)

• Machine Learning for Fluid Mechanics
• Scientific multi-agent reinforcement learning for wall-models of turbulent flows
• Recent advances in applying deep reinforcement learning for flow control: perspectives and future directions
• Learning to school in the presence of hydrodynamic interactions
• A review on Deep Reinforcement Learning for Fluid Mechanics
• A review on deep reinforcement learning for fluid mechanics: an update
• Reinforcement-learning-based control of confined cylinder wakes with stability analyses
• IB2d: a Python and MATLAB implementation of the immersed boundary method
• From Single Neurons to Behavior in the Jellyfish Aurelia aurita
• Reinforcement-learning-based control of convectively-unstable flows
• Active Cloaking in Stokes Flows via Reinforcement Learning
• Swimming strategy of settling elongated microswimmers by reinforcement learning