Temporal Logic Guided Safe Reinforcement Learning Using Control Barrier Functions
arXiv:1903.09885
Abstract
Using reinforcement learning to learn control policies is a challenge when the task is complex with potentially long horizons. Ensuring adequate but safe exploration is also crucial for controlling physical systems. In this paper, we use temporal logic to facilitate specification and learning of complex tasks. We combine temporal logic with control Lyapunov functions to improve exploration. We incorporate control barrier functions to safeguard the exploration and deployment process. We develop a flexible and learnable system that allows users to specify task objectives and constraints in different forms and at various levels. The framework is also able to take advantage of known system dynamics and handle unknown environmental dynamics by integrating model-free learning with model-based planning.
References in corpus (6)
- Leveraging Demonstrations for Deep Reinforcement Learning on Robotics Problems with Sparse Rewards
- Safe Exploration in Continuous Action Spaces
- Barrier-Certified Adaptive Reinforcement Learning with Applications to Brushbot Navigation
- Environment-Independent Task Specifications via GLTL
- Automata Guided Reinforcement Learning With Demonstrations
- Reinforcement Learning With Temporal Logic Rewards
Cited by in corpus (11)
- Reward Machines: Exploiting Reward Function Structure in Reinforcement Learning
- Cautious Reinforcement Learning with Logical Constraints
- ShieldNN: A Provably Safe NN Filter for Unsafe NN Controllers
- Neurosymbolic Reinforcement Learning with Formally Verified Exploration
- Tractable Reinforcement Learning of Signal Temporal Logic Objectives
- Safe-visor Architecture for Sandboxing (AI-based) Unverified Controllers in Stochastic Cyber-Physical Systems
- Safety-Critical Control using Optimal-decay Control Barrier Function with Guaranteed Point-wise Feasibility
- Provably Correct Training of Neural Network Controllers Using Reachability Analysis
- Model-Free Learning of Safe yet Effective Controllers
- Using Logical Specifications of Objectives in Multi-Objective Reinforcement Learning
- Failing with Grace: Learning Neural Network Controllers that are Boundedly Unsafe