Continuous Inverse Optimal Control with Locally Optimal Examples

Abstract

Inverse optimal control, also known as inverse reinforcement learning, is the problem of recovering an unknown reward function in a Markov decision process from expert demonstrations of the optimal policy. We introduce a probabilistic inverse optimal control algorithm that scales gracefully with task dimensionality, and is suitable for large, continuous domains where even computing a full policy is impractical. By using a local approximation of the reward function, our method can also drop the assumption that the demonstrations are globally optimal, requiring only local optimality. This allows it to learn from examples that are unsuitable for prior methods.

ICML2012

Cited by in corpus (29)

• An Algorithmic Perspective on Imitation Learning
• Learning Trajectory Preferences for Manipulators via Iterative Improvement
• Freeway Merging in Congested Traffic based on Multipolicy Decision Making with Passive Actor Critic
• Constrained Inverse Optimal Control with Application to a Human Manipulation Task
• Enabling Robots to Communicate their Objectives
• A human factors approach to validating driver models for interaction-aware automated vehicles
• Learning Implicit Priors for Motion Optimization
• Learning MPC for Interaction-Aware Autonomous Driving: A Game-Theoretic Approach
• Calibration of Human Driving Behavior and Preference Using Naturalistic Traffic Data
• Generative Adversarial Imitation Learning for End-to-End Autonomous Driving on Urban Environments
• Rethinking Trajectory Forecasting Evaluation
• Inverse Dynamic Games Based on Maximum Entropy Inverse Reinforcement Learning
• Driving Behavior Modeling using Naturalistic Human Driving Data with Inverse Reinforcement Learning
• Interpretable Modelling of Driving Behaviors in Interactive Driving Scenarios based on Cumulative Prospect Theory
• Human-in-the-Loop Methods for Data-Driven and Reinforcement Learning Systems
• Imitative Planning using Conditional Normalizing Flow
• Robust Reinforcement Learning: A Case Study in Linear Quadratic Regulation
• Literal or Pedagogic Human? Analyzing Human Model Misspecification in Objective Learning
• Expressing Diverse Human Driving Behavior with Probabilistic Rewards and Online Inference
• Generic Prediction Architecture Considering both Rational and Irrational Driving Behaviors
• Behavior Planning of Autonomous Cars with Social Perception
• Courteous Autonomous Cars
• Efficient Sampling-Based Maximum Entropy Inverse Reinforcement Learning with Application to Autonomous Driving
• Imitation Learning via Simultaneous Optimization of Policies and Auxiliary Trajectories
• Inverse Reinforcement Learning in a Continuous State Space with Formal Guarantees
• A Robustness Analysis of Inverse Optimal Control of Bipedal Walking
• Exploiting Variable Impedance for Energy Efficient Sequential Movements
• Diversity in Action: General-Sum Multi-Agent Continuous Inverse Optimal Control
• Robust Dual View Deep Agent