Safe Exploration in Model-based Reinforcement Learning using Control Barrier Functions

Abstract

This paper develops a model-based reinforcement learning (MBRL) framework for learning online the value function of an infinite-horizon optimal control problem while obeying safety constraints expressed as control barrier functions (CBFs). Our approach is facilitated by the development of a novel class of CBFs, termed Lyapunov-like CBFs (LCBFs), that retain the beneficial properties of CBFs for developing minimally-invasive safe control policies while also possessing desirable Lyapunov-like qualities such as positive semi-definiteness. We show how these LCBFs can be used to augment a learning-based control policy to guarantee safety and then leverage this approach to develop a safe exploration framework in a MBRL setting. We demonstrate that our approach can handle more general safety constraints than comparative methods via numerical examples.

arXiv commentAccepted for publication in Automatica

References in corpus (7)

• Control Barrier Function Based Quadratic Programs for Safety Critical Systems
• Data-driven Economic NMPC using Reinforcement Learning
• Adaptive Safety with Control Barrier Functions
• Concurrent learning-based approximate optimal regulation
• Integral Concurrent Learning: Adaptive Control with Parameter Convergence without PE or State Derivatives
• Efficient model-based reinforcement learning for approximate online optimal
• State Following (StaF) Kernel Functions for Function Approximation

Cited by in corpus (2)

• A Survey on Physics Informed Reinforcement Learning: Review and Open Problems
• Lagrangian-based online safe reinforcement learning for state-constrained systems