Intrinsic noise in game dynamical learning
arXiv:0910.4022 · doi:10.1103/PhysRevLett.103.198702
Abstract
Demographic noise has profound effects on evolutionary and population dynamics, as well as on chemical reaction systems and models of epidemiology. Such noise is intrinsic and due to the discreteness of the dynamics in finite populations. We here show that similar noise-sustained trajectories arise in game dynamical learning, where the stochasticity has a different origin: agents sample a finite number of moves of their opponents in-between adaptation events. The limit of infinite batches results in deterministic modified replicator equations, whereas finite sampling leads to a stochastic dynamics. The characteristics of these fluctuations can be computed analytically using methods from statistical physics, and such noise can affect the attractors significantly, leading to noise-sustained cycling or removing periodic orbits of the standard replicator dynamics.
5 pages, 4 figures, accepted by Phys. Rev. Letters
References in corpus (4)
Cited by in corpus (35)
- Cyclic dominance in evolutionary games: A review
- Bacterial growth: a statistical physicist's guide
- Social cycling and conditional responses in the Rock-Paper-Scissors game
- Dynamics of Boltzmann Q-Learning in Two-Player Two-Action Games
- Deterministic limit of temporal difference reinforcement learning for stochastic games
- Entropy production of cyclic population dynamics
- Evolutionary dynamics, intrinsic noise and cycles of co-operation
- Perspectives on adaptive dynamical systems
- Evolutionary and Population Dynamics: A Coupled Approach
- Cycle frequency in standard Rock-Paper-Scissors games: Evidence from experimental economics
- Chaos and unpredictability in evolutionary dynamics in discrete time
- Intrinsic fluctuations of reinforcement learning promote cooperation
- Mobility, fitness collection, and the breakdown of cooperation
- Cycles of cooperation and defection in imperfect learning
- Fluctuation spectra of large random dynamical systems reveal hidden structure in ecological networks
- Modeling the effects of environmental and perceptual uncertainty using deterministic reinforcement learning dynamics with partial observability
- Noise-driven oscillations in microbial population dynamics
- Replicator equations induced by microscopic processes in nonoverlapping population playing bimatrix games
- Balancing selfishness and norm conformity can explain human behavior in large-scale Prisoner's Dilemma games and can poise human groups near criticality
- Learning to play public good games
- Unlearnable Games and "Satisficing'' Decisions: A Simple Model for a Complex World
- Fixation and escape times in stochastic game learning
- Stochastic evolution in populations of ideas
- Incentive and stability in the Rock-Paper-Scissors game: an experimental investigation
- Effects of noise on convergent game learning dynamics
- Coordination problems on networks revisited: statics and dynamics
- Fence-sitters Protect Cooperation in Complex Networks
- Memory and limit cycles in rock-scissors-paper
- Amplitude death in coupled replicator map lattice: averting migration dilemma
- Unstable Dynamics of Adaptation in Unknown Environment due to Novelty Seeking
- Complex dynamics in learning complicated games
- Stochastic Thermodynamics of Non-reciprocally Interacting Particles and Fields
- Recursive Markov Process for Iterated Games with Markov Strategies
- Fluctuations in meta-population exclusion processes
- Towards a taxonomy of learning dynamics in 2 x 2 games