Sample Efficient Path Integral Control under Uncertainty

Abstract

We present a data-driven optimal control framework that can be viewed as a generalization of the path integral (PI) control approach. We find iterative feedback control laws without parameterization based on probabilistic representation of learned dynamics model. The proposed algorithm operates in a forward-backward manner which differentiate from other PI-related methods that perform forward sampling to find optimal controls. Our method uses significantly less samples to find optimal controls compared to other approaches within the PI control family that relies on extensive sampling from given dynamics models or trials on physical systems in model-free fashions. In addition, the learned controllers can be generalized to new tasks without re-sampling based on the compositionality theory for the linearly-solvable optimal control framework. We provide experimental results on three different systems and comparisons with state-of-the-art model-based methods to demonstrate the efficiency and generalizability of the proposed framework.

References in corpus (4)

• Trust Region Policy Optimization
• Gaussian Processes for Data-Efficient Learning in Robotics and Control
• Path Integral Policy Improvement with Covariance Matrix Adaptation
• Path Integral Control by Reproducing Kernel Hilbert Space Embedding

Cited by in corpus (5)

• Bayesian Learning-Based Adaptive Control for Safety Critical Systems
• On Entropy Regularized Path Integral Control for Trajectory Optimization
• Distributed Algorithms for Linearly-Solvable Optimal Control in Networked Multi-Agent Systems
• Cooperative Path Integral Control for Stochastic Multi-Agent Systems
• Compositionality of Linearly Solvable Optimal Control in Networked Multi-Agent Systems