Multi-agent online learning in time-varying games

Abstract

We examine the long-run behavior of multi-agent online learning in games that evolve over time. Specifically, we focus on a wide class of policies based on mirror descent, and we show that the induced sequence of play (a) converges to Nash equilibrium in time-varying games that stabilize in the long run to a strictly monotone limit; and (b) it stays asymptotically close to the evolving equilibrium of the sequence of stage games (assuming they are strongly monotone). Our results apply to both gradient-based and payoff-based feedback - i.e., the "bandit feedback" case where players only get to observe the payoffs of their chosen actions.

35 pages

References in corpus (4)

• Multiplicative Weights Update with Constant Step-Size in Congestion Games: Convergence, Limit Cycles and Chaos
• Distributed stochastic optimization via matrix exponential learning
• Parallelized Structures for MIMO FBMC under Strong Channel Frequency Selectivity
• Bandit learning in concave $N$-person games

Cited by in corpus (6)

• Dynamic Regret of Policy Optimization in Non-stationary Environments
• Stochastic Relaxed Inertial Forward-Backward-Forward splitting for Monotone Inclusions in Hilbert spaces
• Online non-convex optimization with imperfect feedback
• Pricing Mechanism for Resource Sustainability in Competitive Online Learning Multi-Agent Systems
• Memory-Constrained No-Regret Learning in Adversarial Bandits
• Distributed No-Regret Learning in Multi-Agent Systems