Representation of Reinforcement Learning Policies in Reproducing Kernel Hilbert Spaces
arXiv:2002.02863
Abstract
We propose a general framework for policy representation for reinforcement learning tasks. This framework involves finding a low-dimensional embedding of the policy on a reproducing kernel Hilbert space (RKHS). The usage of RKHS based methods allows us to derive strong theoretical guarantees on the expected return of the reconstructed policy. Such guarantees are typically lacking in black-box models, but are very desirable in tasks requiring stability. We conduct several experiments on classic RL domains. The results confirm that the policies can be robustly embedded in a low-dimensional space while the embedded policy incurs almost no decrease in return.
References in corpus (14)
- Continuous control with deep reinforcement learning
- Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor
- On a Formal Model of Safe and Scalable Self-driving Cars
- Robust Adversarial Reinforcement Learning
- Safe Model-based Reinforcement Learning with Stability Guarantees
- Verifiable Reinforcement Learning via Policy Extraction
- Efficient Optimal Learning for Contextual Bandits
- Safe Policy Improvement by Minimizing Robust Baseline Regret
- Practical Contextual Bandits with Regression Oracles
- Hilbert Space Embeddings of POMDPs
- On the Power of Truncated SVD for General High-rank Matrix Estimation Problems
- Attraction-Repulsion Actor-Critic for Continuous Control Reinforcement Learning
- An inequality for the trace of matrix products, using absolute values
- Fourier Policy Gradients