Regularized Inverse Reinforcement Learning
arXiv:2010.03691
Abstract
Inverse Reinforcement Learning (IRL) aims to facilitate a learner's ability to imitate expert behavior by acquiring reward functions that explain the expert's decisions. Regularized IRL applies strongly convex regularizers to the learner's policy in order to avoid the expert's behavior being rationalized by arbitrary constant rewards, also known as degenerate solutions. We propose tractable solutions, and practical methods to obtain them, for regularized IRL. Current methods are restricted to the maximum-entropy IRL framework, limiting them to Shannon-entropy regularizers, as well as proposing the solutions that are intractable in practice. We present theoretical backing for our proposed IRL method's applicability for both discrete and continuous controls, empirically validating our performance on a variety of tasks.
26 pages, 7 figures
References in corpus (15)
- Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor
- Trust Region Policy Optimization
- Guided Cost Learning: Deep Inverse Optimal Control via Policy Optimization
- A Connection between Generative Adversarial Networks, Inverse Reinforcement Learning, and Energy-Based Models
- Adversarial Variational Bayes: Unifying Variational Autoencoders and Generative Adversarial Networks
- A Theory of Regularized Markov Decision Processes
- Learning to Drive using Inverse Reinforcement Learning and Deep Q-Networks
- rlpyt: A Research Code Base for Deep Reinforcement Learning in PyTorch
- GenDICE: Generalized Offline Estimation of Stationary Values
- Primal Wasserstein Imitation Learning
- Imitation Learning as -Divergence Minimization
- Tsallis Reinforcement Learning: A Unified Framework for Maximum Entropy Reinforcement Learning
- A Regularized Approach to Sparse Optimal Policy in Reinforcement Learning
- Ambiguity set and learning via Bregman and Wasserstein
- Maximum Causal Tsallis Entropy Imitation Learning