Hidden Parameter Markov Decision Processes: A Semiparametric Regression Approach for Discovering Latent Task Parametrizations

Abstract

Control applications often feature tasks with similar, but not identical, dynamics. We introduce the Hidden Parameter Markov Decision Process (HiP-MDP), a framework that parametrizes a family of related dynamical systems with a low-dimensional set of latent factors, and introduce a semiparametric regression approach for learning its structure from data. In the control setting, we show that a learned HiP-MDP rapidly identifies the dynamics of a new task instance, allowing an agent to flexibly adapt to task variations.

References in corpus (3)

• Gaussian Process Regression Networks
• Efficient Bayes-Adaptive Reinforcement Learning using Sample-Based Search
• Learning Parameterized Skills

Cited by in corpus (12)

• A Survey of Zero-shot Generalisation in Deep Reinforcement Learning
• Meta Reinforcement Learning with Latent Variable Gaussian Processes
• Importance Weighted Transfer of Samples in Reinforcement Learning
• Augmented World Models Facilitate Zero-Shot Dynamics Generalization From a Single Offline Environment
• Causal Curiosity: RL Agents Discovering Self-supervised Experiments for Causal Representation Learning
• AdaRL: What, Where, and How to Adapt in Transfer Reinforcement Learning
• POMDP-lite for Robust Robot Planning under Uncertainty
• Agent Modelling under Partial Observability for Deep Reinforcement Learning
• Block Contextual MDPs for Continual Learning
• Meta Learning MPC using Finite-Dimensional Gaussian Process Approximations
• Accelerating the Computation of UCB and Related Indices for Reinforcement Learning
• PantheonRL: A MARL Library for Dynamic Training Interactions