Evolution of cooperation in the public goods game with Q-learning

Abstract

Recent paradigm shifts from imitation learning to reinforcement learning (RL) is shown to be productive in understanding human behaviors. In the RL paradigm, individuals search for optimal strategies through interaction with the environment to make decisions. This implies that gathering, processing, and utilizing information from their surroundings are crucial. However, existing studies typically study pairwise games such as the prisoners' dilemma and employ a self-regarding setup, where individuals play against one opponent based solely on their own strategies, neglecting the environmental information. In this work, we investigate the evolution of cooperation with the multiplayer game -- the public goods game using the Q-learning algorithm by leveraging the environmental information. Specifically, the decision-making of players is based upon the cooperation information in their neighborhood. Our results show that cooperation is more likely to emerge compared to the case of imitation learning by using Fermi rule. Of particular interest is the observation of an anomalous non-monotonic dependence which is revealed when voluntary participation is further introduced. The analysis of the Q-table explains the mechanisms behind the cooperation evolution. Our findings indicate the fundamental role of environment information in the RL paradigm to understand the evolution of cooperation, and human behaviors in general.

16 pages, 12 figures, comments are appreciated

References in corpus (12)

• Statistical physics of human cooperation
• Evolutionary dynamics of group interactions on structured populations: A review
• Social diversity and promotion of cooperation in the spatial prisoner's dilemma game
• Reward and cooperation in the spatial public goods game
• Promoting cooperation in social dilemmas via simple coevolutionary rules
• Restricted connections among distinguished players support cooperation
• Numerical analysis of a reinforcement learning model with the dynamic aspiration level in the iterated Prisoner's Dilemma
• The self-organizing impact of averaged payoffs on the evolution of cooperation
• Blocking defector invasion by focusing on the most successful partner
• Cooperation in regular lattices
• Emergence of Cooperation in Two-agent Repeated Games with Reinforcement Learning
• Decoding trust: A reinforcement learning perspective

Cited by in corpus (5)

• Higher-order evolutionary dynamics with game transitions
• Decoding fairness: a reinforcement learning perspective
• Evolution of cooperation with Q-learning: the impact of information perception
• Optimal coordination of resources: A solution from reinforcement learning
• Evolution of cooperation in a bimodal mixture of conditional cooperators