Solving Structured Hierarchical Games Using Differential Backward Induction
arXiv:2106.04663
Abstract
From large-scale organizations to decentralized political systems, hierarchical strategic decision making is commonplace. We introduce a novel class of structured hierarchical games (SHGs) that formally capture such hierarchical strategic interactions. In an SHG, each player is a node in a tree, and strategic choices of players are sequenced from root to leaves, with root moving first, followed by its children, then followed by their children, and so on until the leaves. A player's utility in an SHG depends on its own decision, and on the choices of its parent and all the tree leaves. SHGs thus generalize simultaneous-move games, as well as Stackelberg games with many followers. We leverage the structure of both the sequence of player moves as well as payoff dependence to develop a gradient-based back propagation-style algorithm, which we call Differential Backward Induction (DBI), for approximating equilibria of SHGs. We provide a sufficient condition for convergence of DBI and demonstrate its efficacy in finding approximate equilibrium solutions to several SHG models of hierarchical policy-making problems.
The short version of this paper appears in the proceedings of UAI-22
References in corpus (9)
- Deep Equilibrium Models
- Meta-Learning with Implicit Gradients
- Differentiable Convex Optimization Layers
- On Differentiating Parameterized Argmin and Argmax Problems with Application to Bi-level Optimization
- End to end learning and optimization on graphs
- A Game Theoretic Framework for Model Based Reinforcement Learning
- Linear Lower Bounds and Conditioning of Differentiable Games
- Stochastic Hamiltonian Gradient Methods for Smooth Games
- End-to-End Learning and Intervention in Games