Smooth Imitation Learning for Online Sequence Prediction

Abstract

We study the problem of smooth imitation learning for online sequence prediction, where the goal is to train a policy that can smoothly imitate demonstrated behavior in a dynamic and continuous environment in response to online, sequential context input. Since the mapping from context to behavior is often complex, we take a learning reduction approach to reduce smooth imitation learning to a regression problem using complex function classes that are regularized to ensure smoothness. We present a learning meta-algorithm that achieves fast and stable convergence to a good policy. Our approach enjoys several attractive properties, including being fully deterministic, employing an adaptive learning rate that can provably yield larger policy improvements compared to previous approaches, and the ability to ensure stable convergence. Our empirical results demonstrate significant performance gains over previous approaches.

ICML 2016

References in corpus (1)

• Learning Trajectory Preferences for Manipulators via Iterative Improvement

Cited by in corpus (10)

• A Survey of Label-noise Representation Learning: Past, Present and Future
• Learning for Safety-Critical Control with Control Barrier Functions
• Batch Policy Learning under Constraints
• Strictly Batch Imitation Learning by Energy-based Distribution Matching
• Imitation-Projected Programmatic Reinforcement Learning
• Robust Regression for Safe Exploration in Control
• Neural-Swarm2: Planning and Control of Heterogeneous Multirotor Swarms using Learned Interactions
• Neural-Swarm: Decentralized Close-Proximity Multirotor Control Using Learned Interactions
• Smooth Imitation Learning via Smooth Costs and Smooth Policies
• Dynamic Regret Convergence Analysis and an Adaptive Regularization Algorithm for On-Policy Robot Imitation Learning