Learning to flock through reinforcement

Abstract

Flocks of birds, schools of fish, insects swarms are examples of coordinated motion of a group that arises spontaneously from the action of many individuals. Here, we study flocking behavior from the viewpoint of multi-agent reinforcement learning. In this setting, a learning agent tries to keep contact with the group using as sensory input the velocity of its neighbors. This goal is pursued by each learning individual by exerting a limited control on its own direction of motion. By means of standard reinforcement learning algorithms we show that: i) a learning agent exposed to a group of teachers, i.e. hard-wired flocking agents, learns to follow them, and ii) that in the absence of teachers, a group of independently learning agents evolves towards a state where each agent knows how to flock. In both scenarios, i) and ii), the emergent policy (or navigation strategy) corresponds to the polar velocity alignment mechanism of the well-known Vicsek model. These results show that a) such a velocity alignment may have naturally evolved as an adaptive behavior that aims at minimizing the rate of neighbor loss, and b) prove that this alignment does not only favor (local) polar order, but it corresponds to best policy/strategy to keep group cohesion when the sensory input is limited to the velocity of neighboring agents. In short, to stay together, steer together.

References in corpus (8)

• Novel type of phase transition in a system of self-driven particles
• Collective motion
• Interaction Ruling Animal Collective Behaviour Depends on Topological rather than Metric Distance: Evidence from a Field Study
• Statistical mechanics for natural flocks of birds
• Self-organised complex aerial displays of thousands of starlings: a model
• Flocking and turning: a new model for self-organized collective motion
• The Role of Projection in the Control of Bird Flocks
• Large-scales patterns in a minimal cognitive flocking model: incidental leaders, nematic patterns, and aggregates

Cited by in corpus (16)

• Learning to Control Active Matter
• Reinforcement learning of optimal active particle navigation
• Predator-prey survival pressure is sufficient to evolve swarming behaviors
• Development of swarm behavior in artificial learning agents that adapt to different foraging environments
• Hunting active Brownian particles: Learning optimal behavior
• Environmental path-entropy and collective motion
• Optimal collision avoidance in swarms of active Brownian particles
• Dynamic control of self-assembly of quasicrystalline structures through reinforcement learning
• Run-and-tumble chemotaxis using reinforcement learning
• Arbitrage Equilibrium, Invariance, and the Emergence of Spontaneous Order in the Dynamics of Birds Flocking
• Active Matter Flocking via Predictive Alignment
• Demixing of two species via reciprocally concentration-dependent diffusivity
• Reinforcement Learning for Active Matter
• Kinetic theory of decentralized learning for smart active matter
• Reinforcement Learning reveals fundamental limits on the mixing of active particles
• Tailoring the overlap distribution in driven mean-field spin models