Bayesian multitask inverse reinforcement learning

Abstract

We generalise the problem of inverse reinforcement learning to multiple tasks, from multiple demonstrations. Each one may represent one expert trying to solve a different task, or as different experts trying to solve the same task. Our main contribution is to formalise the problem as statistical preference elicitation, via a number of structured priors, whose form captures our biases about the relatedness of different tasks or expert policies. In doing so, we introduce a prior on policy optimality, which is more natural to specify. We show that our framework allows us not only to learn to efficiently from multiple experts but to also effectively differentiate between the goals of each. Possible applications include analysing the intrinsic motivations of subjects in behavioural experiments and learning from multiple teachers.

Corrected version. 13 pages, 8 figures

Cited by in corpus (26)

• Bayesian Reinforcement Learning: A Survey
• Multi-agent Inverse Reinforcement Learning for Two-person Zero-sum Games
• On the Feasibility of Learning, Rather than Assuming, Human Biases for Reward Inference
• Multi-task Maximum Entropy Inverse Reinforcement Learning
• Learning Shared Representations in Multi-task Reinforcement Learning
• Inverse Reinforcement Learning from Summary Data
• Scalable Bayesian Inverse Reinforcement Learning
• Inverse Reinforcement Learning via Nonparametric Spatio-Temporal Subgoal Modeling
• Learning a Prior over Intent via Meta-Inverse Reinforcement Learning
• Meta Inverse Reinforcement Learning via Maximum Reward Sharing for Human Motion Analysis
• The MAGICAL Benchmark for Robust Imitation
• Learning a Multi-Modal Policy via Imitating Demonstrations with Mixed Behaviors
• Lifelong Inverse Reinforcement Learning
• SIRL: Similarity-based Implicit Representation Learning
• A Function Approximation Method for Model-based High-Dimensional Inverse Reinforcement Learning
• Robust Bayesian reinforcement learning through tight lower bounds
• PsiPhi-Learning: Reinforcement Learning with Demonstrations using Successor Features and Inverse Temporal Difference Learning
• Inverse Reinforcement Learning in Large State Spaces via Function Approximation
• A Generalized Apprenticeship Learning Framework for Capturing Evolving Student Pedagogical Strategies
• Probabilistic inverse reinforcement learning in unknown environments
• Bellman Gradient Iteration for Inverse Reinforcement Learning
• LiMIIRL: Lightweight Multiple-Intent Inverse Reinforcement Learning
• Maximum Entropy Multi-Task Inverse RL
• Probabilistic inverse reinforcement learning in unknown environments
• A Hierarchical Bayesian model for Inverse RL in Partially-Controlled Environments
• Preference elicitation and inverse reinforcement learning