Oscillatory evolution of collective behavior in evolutionary games played with reinforcement learning

Abstract

Large-scale cooperation underpins the evolution of ecosystems and the human society, and the collective behaviors by self-organization of multi-agent systems are the key for understanding. As artificial intelligence (AI) prevails in almost all branches of science, it would be of great interest to see what new insights of collective behavior could be obtained from a multi-agent AI system. Here, we introduce a typical reinforcement learning (RL) algorithm -- Q learning into evolutionary game dynamics, where agents pursue optimal action on the basis of the introspectiveness rather than the birth-death or imitation processes in the traditional evolutionary game (EG). We investigate the cooperation prevalence numerically for a general $2\times 2$ game setting. We find that the cooperation prevalence in the multi-agent AI is amazing of an equal level as in the traditional EG in most cases. However, in the snowdrift games with RL we also reveal that explosive cooperation appears in the form of periodic oscillation, and we study the impact of the payoff structure on its emergence. Finally, we show that the periodic oscillation can also be observed in some other EGs with the RL algorithm, such as the rock-paper-scissors game. Our results offer a reference point to understand emergence of cooperation and oscillatory behaviors in nature and society from AI's perspective.

arXiv comment10 pages, 7 figures

References in corpus (16)

• Statistical physics of human cooperation
• Joint Training of a Convolutional Network and a Graphical Model for Human Pose Estimation
• Using Deep Learning and Google Street View to Estimate the Demographic Makeup of the US
• The outbreak of cooperation among success-driven individuals under noisy conditions
• Memory-Based Snowdrift Game on Networks
• Effect of spatial structure on the evolution of cooperation
• The collapse of cooperation in evolving games
• Emergence of Compositional Representations in Restricted Boltzmann Machines
• Boosting cooperation by involving extortion in spatial Prisoner's dilemma
• Conformity Hinders the Evolution of Cooperation on Scale-Free Networks
• Maximum likelihood reconstruction for Ising models with asynchronous updates
• Simple property of heterogeneous aspiration dynamics: Beyond weak selection
• Coexistence of cooperators and defectors in well mixed populations mediated by limiting resources
• Evolution of Cooperation on Spatially Embedded Networks
• Balancing selfishness and norm conformity can explain human behavior in large-scale Prisoner's Dilemma games and can poise human groups near criticality
• Topology-dependent rationality and quantal response equilibria in structured populations