On Passivity, Reinforcement Learning and Higher-Order Learning in Multi-Agent Finite Games

Abstract

In this paper, we propose a passivity-based methodology for analysis and design of reinforcement learning in multi-agent finite games. Starting from a known exponentially-discounted reinforcement learning scheme, we show that convergence to a Nash distribution can be shown in the class of games characterized by the monotonicity property of their (negative) payoff. We further exploit passivity to propose a class of higher-order schemes that preserve convergence properties, can improve the speed of convergence and can even converge in cases whereby their first-order counterpart fail to converge. We demonstrate these properties through numerical simulations for several representative games.

arXiv comment14 pages, 19 figures. This work has been submitted to the IEEE for possible publication

References in corpus (4)

• A Passivity-Based Approach to Nash Equilibrium Seeking over Networks
• Duality and Network Theory in Passivity-based Cooperative Control
• Dynamics of Boltzmann Q-Learning in Two-Player Two-Action Games
• On imitation dynamics in potential population games

Cited by in corpus (4)

• A Survey on Physics Informed Reinforcement Learning: Review and Open Problems
• Continuous-time Discounted Mirror-Descent Dynamics in Monotone Concave Games
• Dynamic Population Games: A Tractable Intersection of Mean-Field Games and Population Games
• Nash Equilibrium Learning In Large Populations With First-Order Payoff Modifications