Hilbert Space Embeddings of POMDPs
arXiv:1210.4887
Abstract
A nonparametric approach for policy learning for POMDPs is proposed. The approach represents distributions over the states, observations, and actions as embeddings in feature spaces, which are reproducing kernel Hilbert spaces. Distributions over states given the observations are obtained by applying the kernel Bayes' rule to these distribution embeddings. Policies and value functions are defined on the feature space over states, which leads to a feature space expression for the Bellman equation. Value iteration may then be used to estimate the optimal value function and associated policy. Experimental results confirm that the correct policy is learned using the feature space representation.
Appears in Proceedings of the Twenty-Eighth Conference on Uncertainty in Artificial Intelligence (UAI2012)
References in corpus (6)
Cited by in corpus (12)
- Kernel Mean Embedding of Distributions: A Review and Beyond
- Learning Decentralized Controllers for Robot Swarms with Graph Neural Networks
- Barrier-Certified Adaptive Reinforcement Learning with Applications to Brushbot Navigation
- A Measure-Theoretic Approach to Kernel Conditional Mean Embeddings
- Characteristic Kernels and Infinitely Divisible Distributions
- Kernel Bayesian Inference with Posterior Regularization
- A New Distribution-Free Concept for Representing, Comparing, and Propagating Uncertainty in Dynamical Systems with Kernel Probabilistic Programming
- Kernel Autocovariance Operators of Stationary Processes: Estimation and Convergence
- Filtering with State-Observation Examples via Kernel Monte Carlo Filter
- Representation of Reinforcement Learning Policies in Reproducing Kernel Hilbert Spaces
- Continuous-time Value Function Approximation in Reproducing Kernel Hilbert Spaces
- Model-based Kernel Sum Rule: Kernel Bayesian Inference with Probabilistic Models