Safe Exploration of State and Action Spaces in Reinforcement Learning

Abstract

In this paper, we consider the important problem of safe exploration in reinforcement learning. While reinforcement learning is well-suited to domains with complex transition dynamics and high-dimensional state-action spaces, an additional challenge is posed by the need for safe and efficient exploration. Traditional exploration techniques are not particularly useful for solving dangerous tasks, where the trial and error process may lead to the selection of actions whose execution in some states may result in damage to the learning system (or any other system). Consequently, when an agent begins an interaction with a dangerous and high-dimensional state-action space, an important question arises; namely, that of how to avoid (or at least minimize) damage caused by the exploration of the state-action space. We introduce the PI-SRL algorithm which safely improves suboptimal albeit robust behaviors for continuous state and action control tasks and which efficiently learns from the experience gained from the environment. We evaluate the proposed method in four complex tasks: automatic car parking, pole-balancing, helicopter hovering, and business management.

References in corpus (1)

• Risk-sensitive Reinforcement Learning

Cited by in corpus (6)

• Exploration in Deep Reinforcement Learning: A Survey
• A Survey on Physics Informed Reinforcement Learning: Review and Open Problems
• Safe global optimization of expensive noisy black-box functions in the $\delta$-Lipschitz framework
• Efficient and Safe Exploration in Deterministic Markov Decision Processes with Unknown Transition Models
• Safe Learning MPC with Limited Model Knowledge and Data
• Safe Online Bid Optimization with Return on Investment and Budget Constraints