SAMBA: Safe Model-Based & Active Reinforcement Learning
arXiv:2006.09436
Abstract
In this paper, we propose SAMBA, a novel framework for safe reinforcement learning that combines aspects from probabilistic modelling, information theory, and statistics. Our method builds upon PILCO to enable active exploration using novel(semi-)metrics for out-of-sample Gaussian process evaluation optimised through a multi-objective problem that supports conditional-value-at-risk constraints. We evaluate our algorithm on a variety of safe dynamical system benchmarks involving both low and high-dimensional state representations. Our results show orders of magnitude reductions in samples and violations compared to state-of-the-art methods. Lastly, we provide intuition as to the effectiveness of the framework by a detailed analysis of our active metrics and safety constraints.
References in corpus (13)
- Playing Atari with Deep Reinforcement Learning
- Trust Region Policy Optimization
- Rainbow: Combining Improvements in Deep Reinforcement Learning
- Learning Latent Dynamics for Planning from Pixels
- Multi-Task Learning as Multi-Objective Optimization
- Safe Model-based Reinforcement Learning with Stability Guarantees
- Uncertainty Estimation Using a Single Deep Deterministic Neural Network
- Safe Exploration in Continuous Action Spaces
- Lyapunov-based Safe Policy Optimization for Continuous Control
- Model-Based Active Exploration
- Algorithms for CVaR Optimization in MDPs
- Variational Gaussian Process Dynamical Systems
- Ready Policy One: World Building Through Active Learning
Cited by in corpus (6)
- Conservative Safety Critics for Exploration
- Are we Forgetting about Compositional Optimisers in Bayesian Optimisation?
- GenSafe: A Generalizable Safety Enhancer for Safe Reinforcement Learning Algorithms Based on Reduced Order Markov Decision Process Model
- DESTA: A Framework for Safe Reinforcement Learning with Markov Games of Intervention
- Risk Sensitive Model-Based Reinforcement Learning using Uncertainty Guided Planning
- Learning Robust Controllers Via Probabilistic Model-Based Policy Search