Solving POMDPs by Searching in Policy Space
arXiv:1301.7380
Abstract
Most algorithms for solving POMDPs iteratively improve a value function that implicitly represents a policy and are said to search in value function space. This paper presents an approach to solving POMDPs that represents a policy explicitly as a finite-state controller and iteratively improves the controller by search in policy space. Two related algorithms illustrate this approach. The first is a policy iteration algorithm that can outperform value iteration in solving infinitehorizon POMDPs. It provides the foundation for a new heuristic search algorithm that promises further speedup by focusing computational effort on regions of the problem space that are reachable, or likely to be reached, from a start state.
Appears in Proceedings of the Fourteenth Conference on Uncertainty in Artificial Intelligence (UAI1998)
References in corpus (1)
Cited by in corpus (8)
- Value-Function Approximations for Partially Observable Markov Decision Processes
- The Complexity of Decentralized Control of Markov Decision Processes
- MAA*: A Heuristic Search Algorithm for Solving Decentralized POMDPs
- Solving POMDPs by Searching the Space of Finite Policies
- Speeding Up the Convergence of Value Iteration in Partially Observable Markov Decision Processes
- Nonapproximability Results for Partially Observable Markov Decision Processes
- My Brain is Full: When More Memory Helps
- Sparse Stochastic Finite-State Controllers for POMDPs