Multi-task Maximum Entropy Inverse Reinforcement Learning

Abstract

Multi-task Inverse Reinforcement Learning (IRL) is the problem of inferring multiple reward functions from expert demonstrations. Prior work, built on Bayesian IRL, is unable to scale to complex environments due to computational constraints. This paper contributes a formulation of multi-task IRL in the more computationally efficient Maximum Causal Entropy (MCE) IRL framework. Experiments show our approach can perform one-shot imitation learning in a gridworld environment that single-task IRL algorithms need hundreds of demonstrations to solve. We outline preliminary work using meta-learning to extend our method to the function approximator setting of modern MCE IRL algorithms. Evaluating on multi-task variants of common simulated robotics benchmarks, we discover serious limitations of these IRL algorithms, and conclude with suggestions for further work.

Presented at 1st Workshop on Goal Specifications for Reinforcement Learning (ICML/IJCAI/AAMAS 2018)

References in corpus (4)

• InfoGAIL: Interpretable Imitation Learning from Visual Demonstrations
• Multi-Agent Generative Adversarial Imitation Learning
• Multi-Modal Imitation Learning from Unstructured Demonstrations using Generative Adversarial Nets
• Repeated Inverse Reinforcement Learning

Cited by in corpus (10)

• Learning to Generalize from Sparse and Underspecified Rewards
• Few-Shot Goal Inference for Visuomotor Learning and Planning
• On the Feasibility of Learning, Rather than Assuming, Human Biases for Reward Inference
• The MAGICAL Benchmark for Robust Imitation
• Generalized Decision Transformer for Offline Hindsight Information Matching
• LiMIIRL: Lightweight Multiple-Intent Inverse Reinforcement Learning
• Maximum Entropy Multi-Task Inverse RL
• Meta-Adversarial Inverse Reinforcement Learning for Decision-making Tasks
• Quickest change detection with unknown parameters: Constant complexity and near optimality
• Combining Reward Information from Multiple Sources