Coupled Replicator Equations for the Dynamics of Learning in Multiagent Systems

Abstract

Starting with a group of reinforcement-learning agents we derive coupled replicator equations that describe the dynamics of collective learning in multiagent systems. We show that, although agents model their environment in a self-interested way without sharing knowledge, a game dynamics emerges naturally through environment-mediated interactions. An application to rock-scissors-paper game interactions shows that the collective learning dynamics exhibits a diversity of competitive and cooperative behaviors. These include quasiperiodicity, stable limit cycles, intermittency, and deterministic chaos--behaviors that should be expected in heterogeneous multiagent systems described by the general replicator equations we derive.

4 pages, 3 figures, http://www.santafe.edu/projects/CompMech/papers/credlmas.html; updated references, corrected typos, changed content

Cited by in corpus (48)

• Coevolutionary Dynamics: From Finite to Infinite Populations
• On the Properties of the Softmax Function with Application in Game Theory and Reinforcement Learning
• Evolutionary game dynamics in a growing structured population
• Cyclic dominance and biodiversity in well-mixed populations
• Dynamics of Boltzmann Q-Learning in Two-Player Two-Action Games
• Dynamics of Bayesian Updating with Dependent Data and Misspecified Models
• Stability and Diversity in Collective Adaptation
• The prevalence of chaotic dynamics in games with many players
• Deterministic limit of temporal difference reinforcement learning for stochastic games
• Stochastic gain in population dynamics
• Perspectives on adaptive dynamical systems
• Intrinsic noise in game dynamical learning
• Cyclic game dynamics driven by iterated reasoning
• Artificial intelligence meets minority game: toward optimal resource allocation
• Ecological communities from random generalised Lotka-Volterra dynamics with non-linear feedback
• The Hidden Fragility of Complex Systems -- Consequences of Change, Changing Consequences
• Cycles of cooperation and defection in imperfect learning
• Weight of fitness deviation governs strict physical chaos in replicator dynamics
• Modeling the effects of environmental and perceptual uncertainty using deterministic reinforcement learning dynamics with partial observability
• Replicator equations induced by microscopic processes in nonoverlapping population playing bimatrix games
• Deciphering chaos in evolutionary games
• Learning to play public good games
• Continuous Strategy Replicator Dynamics for Multi--Agent Learning
• Coevolutionary networks of reinforcement-learning agents
• Scale-free memory model for multiagent reinforcement learning. Mean field approximation and rock-paper-scissors dynamics
• Fixation and escape times in stochastic game learning
• Spontaneous Segregation of Agents Across Double Auction Markets
• Follow-the-Regularized-Leader Routes to Chaos in Routing Games
• Limits on the Evolutionary Rates of Biological Traits
• Dynamical selection of Nash equilibria using Experience Weighted Attraction Learning: emergence of heterogeneous mixed equilibria
• Stochastic evolution in populations of ideas
• Effects of noise on convergent game learning dynamics
• The Replicator Dynamics of Zero-Sum Games Arise from a Novel Poisson Algebra
• Coordination problems on networks revisited: statics and dynamics
• Memory and limit cycles in rock-scissors-paper
• Emergence of Cooperative Long-term Market Loyalty in Double Auction Markets
• Complex behaviour in cyclic competition bimatrix games
• Unstable Dynamics of Adaptation in Unknown Environment due to Novelty Seeking
• Complex dynamics in learning complicated games
• Amplitude death in coupled replicator map lattice: averting migration dilemma
• Exploitation by asymmetry of information reference in coevolutionary learning in prisoner's dilemma game
• On the use of chaotic dynamics for mobile network design and analysis: towards a trace data generator
• On complex adaptive systems and terrorism
• Bifurcation Mechanism Design -- From Optimal Flat Taxes to Improved Cancer Treatments
• On the non-transverse homoclinic channel of a center manifold
• Replicator Dynamics of Co-Evolving Networks
• A Structure-Preserving Rational Integrator for the Replicator Dynamics on the Probability Simplex
• Catastrophe by Design in Population Games: Destabilizing Wasteful Locked-in Technologies