Variational Inference for Data-Efficient Model Learning in POMDPs
arXiv:1805.09281
Abstract
Partially observable Markov decision processes (POMDPs) are a powerful abstraction for tasks that require decision making under uncertainty, and capture a wide range of real world tasks. Today, effective planning approaches exist that generate effective strategies given black-box models of a POMDP task. Yet, an open question is how to acquire accurate models for complex domains. In this paper we propose DELIP, an approach to model learning for POMDPs that utilizes amortized structured variational inference. We empirically show that our model leads to effective control strategies when coupled with state-of-the-art planners. Intuitively, model-based approaches should be particularly beneficial in environments with changing reward structures, or where rewards are initially unknown. Our experiments confirm that DELIP is particularly effective in this setting.
References in corpus (12)
- Adam: A Method for Stochastic Optimization
- Neural Discrete Representation Learning
- Stochastic Backpropagation and Approximate Inference in Deep Generative Models
- Action-Conditional Video Prediction using Deep Networks in Atari Games
- Imagination-Augmented Agents for Deep Reinforcement Learning
- Hybrid Reward Architecture for Reinforcement Learning
- Sequential Neural Models with Stochastic Layers
- Deep Kalman Filters
- Learning Cooperative Visual Dialog Agents with Deep Reinforcement Learning
- Learning and Querying Fast Generative Models for Reinforcement Learning
- Learning Multimodal Transition Dynamics for Model-Based Reinforcement Learning
- Prediction Under Uncertainty with Error-Encoding Networks