The Confluence of Networks, Games and Learning

Abstract

Recent years have witnessed significant advances in technologies and services in modern network applications, including smart grid management, wireless communication, cybersecurity as well as multi-agent autonomous systems. Considering the heterogeneous nature of networked entities, emerging network applications call for game-theoretic models and learning-based approaches in order to create distributed network intelligence that responds to uncertainties and disruptions in a dynamic or an adversarial environment. This paper articulates the confluence of networks, games and learning, which establishes a theoretical underpinning for understanding multi-agent decision-making over networks. We provide an selective overview of game-theoretic learning algorithms within the framework of stochastic approximation theory, and associated applications in some representative contexts of modern network systems, such as the next generation wireless communication networks, the smart grid and distributed machine learning. In addition to existing research works on game-theoretic learning over networks, we highlight several new angles and research endeavors on learning in games that are related to recent developments in artificial intelligence. Some of the new angles extrapolate from our own research interests. The overall objective of the paper is to provide the reader a clear picture of the strengths and challenges of adopting game-theoretic learning methods within the context of network systems, and further to identify fruitful future research directions on both theoretical and applied studies.

arXiv commentThe manuscript has been published in IEEE Control System Magazine as part of the special issue "Distributed Nash Equilibrium Seeking over Networks" Update note: fixed typos

References in corpus (20)

• Federated Optimization: Distributed Machine Learning for On-Device Intelligence
• Learning to Communicate with Deep Multi-Agent Reinforcement Learning
• A Differential Equation for Modeling Nesterov's Accelerated Gradient Method: Theory and Insights
• A Variational Perspective on Accelerated Methods in Optimization
• Fully Decentralized Multi-Agent Reinforcement Learning with Networked Agents
• A Survey of Learning in Multiagent Environments: Dealing with Non-Stationarity
• A Dynamic Games Approach to Proactive Defense Strategies against Advanced Persistent Threats in Cyber-Physical Systems
• Multi-Agent Reinforcement Learning via Double Averaging Primal-Dual Optimization
• Modeling, Analysis, and Mitigation of Dynamic Botnet Formation in Wireless IoT Networks
• A unified view of entropy-regularized Markov decision processes
• On the Secure and Reconfigurable Multi-Layer Network Design for Critical Information Dissemination in the Internet of Battlefield Things (IoBT)
• On Finding Local Nash Equilibria (and Only Local Nash Equilibria) in Zero-Sum Games
• On Nonconvex Optimization for Machine Learning: Gradients, Stochasticity, and Saddle Points
• Bandit learning in concave $N$-person games
• Global Convergence of Policy Gradient for Sequential Zero-Sum Linear Quadratic Dynamic Games
• Model-Based Multi-Agent RL in Zero-Sum Markov Games with Near-Optimal Sample Complexity
• Learning Stationary Correlated Equilibria in Constrained General-Sum Stochastic Games
• Derivative-Free Policy Optimization for Linear Risk-Sensitive and Robust Control Design: Implicit Regularization and Sample Complexity
• Blackwell Online Learning for Markov Decision Processes
• Optimal Curing Strategy for Competing Epidemics Spreading over Complex Networks