Autotaxis of a Run-and-Tumble Agent via Reinforcement Learning

Abstract

We study the run-tumble motion of a single autochemotactic agent that secretes and senses chemical cues using a reinforcement learning (RL) framework. In a chemical environment that changes dynamically via secretion, diffusion, and degradation of attractant chemicals, we study the effectiveness of RL for a successful navigation strategy. We show that RL enhances chemotactic performance. Starting from a random initial position, the probability of finding the agent in the favorable region (near the attractant peak) increases faster with time and reaches a higher value in the long time limit when RL is used, compared to the case when the agent performs usual run-tumble chemotaxis without using RL. We also characterize the chemotactic performance of the agent by varying different parameters like rates of secretion, degradation, and diffusion. Importantly, we observe a non-monotonic dependence of the chemotactic performance on the learning and exploration rates, indicating the existence of an optimal RL strategy for the most efficient navigation. We explain these observations as a result of the interplay between the pre-existing attractant environment and the agent-generated attractant profile.

References in corpus (25)

• Strong-coupling dynamics of a multi-cellular chemotactic system
• Learning to flock through reinforcement
• Dynamics of a microorganism moving by chemotaxis in its own secretion
• Self Trapping of a Single Bacterium in its Own Chemoattractant
• Chemotaxis When Bacteria Remember: Drift versus Diffusion
• Surfing along concentration filaments: sperm chemotaxis in physiological shear flows
• Hunting active Brownian particles: Learning optimal behavior
• Reinforcement Learning of Artificial Microswimmers
• Optimal methylation noise for best chemotactic performance of {\sl E. coli}
• Optimal search in E.coli chemotaxis
• Anomalous Collective Dynamics of Auto-Chemotactic Populations
• Learning optimal integration of spatial and temporal information in noisy chemotaxis
• Effect of receptor clustering on chemotactic performance of Escherichia coli: sensing versus adaptation
• Spontaneous trail formation in populations of auto-chemotactic walkers
• Run-and-tumble motion with step-like responses to a stochastic input
• Run-and-tumble chemotaxis using reinforcement learning
• Effect of switching time scale of receptor activity on chemotactic performance of Escherichia coli
• Attack and defence in cellular decision-making: lessons from machine learning
• Effect of receptor cooperativity on methylation dynamics in bacterial chemotaxis with weak and strong gradient
• Short time extremal response to step stimulus for a single cell {\sl E. coli}
• Bacterial Chemotaxis in a Traveling Wave Attractant Environment
• Spontaneous Emergence of Run-and-Tumble-Like Dynamics in a Robotic Analog of \textit{Chlamydomonas}: Experiment and Theory
• Run-and-Tumble Particles Learning Chemotaxis
• Autonomous Discovery of the Ising Model's Critical Parameters with Reinforcement Learning
• Dynamics of chemo-receptor activity with time-periodic attractant field