Learning a Low-dimensional Representation of a Safe Region for Safe Reinforcement Learning on Dynamical Systems
arXiv:2010.09555 · doi:10.1109/TNNLS.2021.3106818
Abstract
For safely applying reinforcement learning algorithms on high-dimensional nonlinear dynamical systems, a simplified system model is used to formulate a safe reinforcement learning framework. Based on the simplified system model, a low-dimensional representation of the safe region is identified and is used to provide safety estimates for learning algorithms. However, finding a satisfying simplified system model for complex dynamical systems usually requires a considerable amount of effort. To overcome this limitation, we propose in this work a general data-driven approach that is able to efficiently learn a low-dimensional representation of the safe region. Through an online adaptation method, the low-dimensional representation is updated by using the feedback data such that more accurate safety estimates are obtained. The performance of the proposed approach for identifying the low-dimensional representation of the safe region is demonstrated with a quadcopter example. The results show that, compared to previous work, a more reliable and representative low-dimensional representation of the safe region is derived, which then extends the applicability of the safe reinforcement learning framework.
References in corpus (9)
- End-to-End Training of Deep Visuomotor Policies
- Benchmarking Deep Reinforcement Learning for Continuous Control
- Safe Reinforcement Learning Using Robust MPC
- Transfer from Simulation to Real World through Learning Deep Inverse Dynamics Model
- Safe Learning of Regions of Attraction for Uncertain, Nonlinear Systems with Gaussian Processes
- Risk-sensitive Reinforcement Learning
- Virtual vs. Real: Trading Off Simulations and Physical Experiments in Reinforcement Learning with Bayesian Optimization
- Safe Exploration in Markov Decision Processes
- Benchmarking Reinforcement Learning Algorithms on Real-World Robots