Mean-Field Control Approach to Decentralized Stochastic Control with Finite-Dimensional Memories
arXiv:2209.05067 · doi:10.3390/e25050791
Abstract
Decentralized stochastic control (DSC) considers the optimal control problem of a multi-agent system. However, DSC cannot be solved except in the special cases because the estimation among the agents is generally intractable. In this work, we propose memory-limited DSC (ML-DSC), in which each agent compresses the observation history into the finite-dimensional memory. Because this compression simplifies the estimation among the agents, ML-DSC can be solved in more general cases based on the mean-field control theory. We demonstrate ML-DSC in the general LQG problem. Because estimation and control are not clearly separated in the general LQG problem, the Riccati equation is modified to the decentralized Riccati equation, which improves estimation as well as control. Our numerical experiment shows that the decentralized Riccati equation is superior to the conventional Riccati equation.
arXiv admin note: text overlap with arXiv:2203.10682
References in corpus (5)
- Optimizing Memory-Bounded Controllers for Decentralized POMDPs
- Anytime Planning for Decentralized POMDPs using Expectation Maximization
- Memory-Limited Partially Observable Stochastic Control and its Mean-Field Control Approach
- Pontryagin's Minimum Principle and Forward-Backward Sweep Method for the System of HJB-FP Equations in Memory-Limited Partially Observable Stochastic Control
- Forward and Backward Bellman equations improve the efficiency of EM algorithm for DEC-POMDP
Cited by in corpus (5)
- Rational social distancing policy during epidemics with limited healthcare capacity
- Theory for Optimal Estimation and Control under Resource Limitations and Its Applications to Biological Information Processing and Decision-Making
- Rational social distancing in epidemics with uncertain vaccination timing
- Resource Limitations induce Phase Transitions in Biological Information Processing
- Theoretical Analysis of Resource-Induced Phase Transitions in Estimation Strategies