Aspiration Learning in Coordination Games

Abstract

We consider the problem of distributed convergence to efficient outcomes in coordination games through dynamics based on aspiration learning. Under aspiration learning, a player continues to play an action as long as the rewards received exceed a specified aspiration level. Here, the aspiration level is a fading memory average of past rewards, and these levels also are subject to occasional random perturbations. A player becomes dissatisfied whenever a received reward is less than the aspiration level, in which case the player experiments with a probability proportional to the degree of dissatisfaction. Our first contribution is the characterization of the asymptotic behavior of the induced Markov chain of the iterated process in terms of an equivalent finite-state Markov chain. We then characterize explicitly the behavior of the proposed aspiration learning in a generalized version of coordination games, examples of which include network formation and common-pool games. In particular, we show that in generic coordination games the frequency at which an efficient action profile is played can be made arbitrarily large. Although convergence to efficient outcomes is desirable, in several coordination games, such as common-pool games, attainability of fair outcomes, i.e., sequences of plays at which players experience highly rewarding returns with the same frequency, might also be of special interest. To this end, we demonstrate through analysis and simulations that aspiration learning also establishes fair outcomes in all symmetric coordination games, including common-pool games.

arXiv comment27 pages

Cited by in corpus (5)

• Empirical Centroid Fictitious Play: An Approach For Distributed Learning In Multi-Agent Games
• Decentralized Learning for Optimality in Stochastic Dynamic Teams and Games with Local Control and Global State Information
• Satisficing Paths and Independent Multi-Agent Reinforcement Learning in Stochastic Games
• From Weak Learning to Strong Learning in Fictitious Play Type Algorithms
• Learning-based Dynamic Pinning of Parallelized Applications in Many-Core Systems