Follow-the-Regularized-Leader Routes to Chaos in Routing Games

Abstract

We study the emergence of chaotic behavior of Follow-the-Regularized Leader (FoReL) dynamics in games. We focus on the effects of increasing the population size or the scale of costs in congestion games, and generalize recent results on unstable, chaotic behaviors in the Multiplicative Weights Update dynamics to a much larger class of FoReL dynamics. We establish that, even in simple linear non-atomic congestion games with two parallel links and any fixed learning rate, unless the game is fully symmetric, increasing the population size or the scale of costs causes learning dynamics to become unstable and eventually chaotic, in the sense of Li-Yorke and positive topological entropy. Furthermore, we show the existence of novel non-standard phenomena such as the coexistence of stable Nash equilibria and chaos in the same game. We also observe the simultaneous creation of a chaotic attractor as another chaotic attractor gets destroyed. Lastly, although FoReL dynamics can be strange and non-equilibrating, we prove that the time average still converges to an exact equilibrium for any choice of learning rate and any scale of costs.

arXiv comment30 pages, 8 figures

References in corpus (10)

• Which Training Methods for GANs do actually Converge?
• Game theory, maximum entropy, minimum discrepancy and robust Bayesian decision theory
• Coupled Replicator Equations for the Dynamics of Learning in Multiagent Systems
• The Unusual Effectiveness of Averaging in GAN Training
• Dynamics of Boltzmann Q-Learning in Two-Player Two-Action Games
• Stability and Diversity in Collective Adaptation
• Multiplicative Weights Update with Constant Step-Size in Congestion Games: Convergence, Limit Cycles and Chaos
• Escort Evolutionary Game Theory
• Entropy computing via integration over fractal measures
• Hessian barrier algorithms for linearly constrained optimization problems