Solving POMDPs by Searching in Policy Space
arXiv:1301.7380
Abstract
Most algorithms for solving POMDPs iteratively improve a value function that implicitly represents a policy and are said to search in value function space. This paper presents an approach to solving POMDPs that represents a policy explicitly as a finite-state controller and iteratively improves the controller by search in policy space. Two related algorithms illustrate this approach. The first is a policy iteration algorithm that can outperform value iteration in solving infinitehorizon POMDPs. It provides the foundation for a new heuristic search algorithm that promises further speedup by focusing computational effort on regions of the problem space that are reachable, or likely to be reached, from a start state.
Appears in Proceedings of the Fourteenth Conference on Uncertainty in Artificial Intelligence (UAI1998)
References in corpus (1)
Cited by in corpus (19)
- Value-Function Approximations for Partially Observable Markov Decision Processes
- Online Planning Algorithms for POMDPs
- Finding Approximate POMDP solutions Through Belief Compression
- The Complexity of Decentralized Control of Markov Decision Processes
- Learning Finite-State Controllers for Partially Observable Environments
- MAA*: A Heuristic Search Algorithm for Solving Decentralized POMDPs
- Solving POMDPs by Searching the Space of Finite Policies
- Speeding Up the Convergence of Value Iteration in Partially Observable Markov Decision Processes
- Nonapproximability Results for Partially Observable Markov Decision Processes
- Learning Policies with External Memory
- Policy Iteration for Decentralized Control of Markov Decision Processes
- Approximate information state for approximate planning and reinforcement learning in partially observed systems
- My Brain is Full: When More Memory Helps
- Sparse Stochastic Finite-State Controllers for POMDPs
- Control Synthesis for Cyber-Physical Systems to Satisfy Metric Interval Temporal Logic Objectives under Timing and Actuator Attacks
- Latent-state models for precision medicine
- Partially Observed, Multi-objective Markov Games
- Bayesian Nonparametric Modelling for Model-Free Reinforcement Learning in LTE-LAA and Wi-Fi Coexistence
- Bayesian Nonparametric Reinforcement Learning in LTE and Wi-Fi Coexistence