Constraint Learning for Control Tasks with Limited Duration Barrier Functions
arXiv:1908.09506 · doi:10.1016/j.automatica.2021.109504
Abstract
When deploying autonomous agents in unstructured environments over sustained periods of time, adaptability and robustness oftentimes outweigh optimality as a primary consideration. In other words, safety and survivability constraints play a key role and in this paper, we present a novel, constraint-learning framework for control tasks built on the idea of constraints-driven control. However, since control policies that keep a dynamical agent within state constraints over infinite horizons are not always available, this work instead considers constraints that can be satisfied over some finite time horizon T > 0, which we refer to as limited-duration safety. Consequently, value function learning can be used as a tool to help us find limited-duration safe policies. We show that, in some applications, the existence of limited-duration safe policies is actually sufficient for long-duration autonomy. This idea is illustrated on a swarm of simulated robots that are tasked with covering a given area, but that sporadically need to abandon this task to charge batteries. We show how the battery-charging behavior naturally emerges as a result of the constraints. Additionally, using a cart-pole simulation environment, we show how a control policy can be efficiently transferred from the source task, balancing the pole, to the target task, moving the cart to one direction without letting the pole fall down.
arXiv comment11 pages, 4 figures; published in Automatica
References in corpus (7)
- Continuous control with deep reinforcement learning
- Control Barrier Function Based Quadratic Programs for Safety Critical Systems
- DeepMind Control Suite
- Barrier-Certified Adaptive Reinforcement Learning with Applications to Brushbot Navigation
- Constrained Deep Networks: Lagrangian Optimization via Log-Barrier Extensions
- Permissive Barrier Certificates for Safe Stabilization Using Sum-of-squares
- Continuous-time Value Function Approximation in Reproducing Kernel Hilbert Spaces