Adaptive Learning in Continuous Games: Optimal Regret Bounds and Convergence to Nash Equilibrium
arXiv:2104.12761
Abstract
In game-theoretic learning, several agents are simultaneously following their individual interests, so the environment is non-stationary from each player's perspective. In this context, the performance of a learning algorithm is often measured by its regret. However, no-regret algorithms are not created equal in terms of game-theoretic guarantees: depending on how they are tuned, some of them may drive the system to an equilibrium, while others could produce cyclic, chaotic, or otherwise divergent trajectories. To account for this, we propose a range of no-regret policies based on optimistic mirror descent, with the following desirable properties: i) they do not require any prior tuning or knowledge of the game; ii) they all achieve O(\sqrt{T}) regret against arbitrary, adversarial opponents; and iii) they converge to the best response against convergent opponents. Also, if employed by all players, then iv) they guarantee O(1) social regret; while v) the induced sequence of play converges to Nash equilibrium with O(1) individual regret in all variationally stable games (a class of games that includes all monotone and convex-concave zero-sum games).
arXiv commentIn the 34th Annual Conference on Learning Theory (COLT 2021); 35 pages, 2 figures
References in corpus (12)
- A Variational Inequality Perspective on Generative Adversarial Networks
- Online Learning with Predictable Sequences
- Optimistic mirror descent in saddle-point problems: Going the extra (gradient) mile
- Fast Convergence of Regularized Learning in Games
- Multiplicative Weights Update with Constant Step-Size in Congestion Games: Convergence, Limit Cycles and Chaos
- No-regret Dynamics and Fictitious Play
- Explore Aggressively, Update Conservatively: Stochastic Extragradient Methods with Variable Stepsize Scaling
- On the convergence of single-call stochastic extra-gradient methods
- Faster Rates for Convex-Concave Games
- Online mirror descent and dual averaging: keeping pace in the dynamic case
- Adaptive extra-gradient methods for min-max optimization and games
- Hedging in games: Faster convergence of external and swap regrets