Black-Box Policy Search with Probabilistic Programs
arXiv:1507.04635
Abstract
In this work, we explore how probabilistic programs can be used to represent policies in sequential decision problems. In this formulation, a probabilistic program is a black-box stochastic simulator for both the problem domain and the agent. We relate classic policy gradient techniques to recently introduced black-box variational methods which generalize to probabilistic program inference. We present case studies in the Canadian traveler problem, Rock Sample, and a benchmark for optimal diagnosis inspired by Guess Who. Each study illustrates how programs can efficiently represent policies using moderate numbers of parameters.
References in corpus (7)
- Multiple Object Recognition with Visual Attention
- Church: a language for generative models
- Path integrals and symmetry breaking for optimal control theory
- Optimal control as a graphical model inference problem
- Automated Variational Inference in Probabilistic Programming
- Selecting Computations: Theory and Applications
- New inference strategies for solving Markov Decision Processes using reversible jump MCMC
Cited by in corpus (10)
- Quantum synchronization and quantum state sharing in irregular complex network
- The Generalized Reparameterization Gradient
- Overdispersed Black-Box Variational Inference
- Nested Reasoning About Autonomous Agents Using Probabilistic Programs
- Bayesian Optimization for Probabilistic Programs
- Probabilistic Programs with Stochastic Conditioning
- Bayesian Policy Search for Stochastic Domains
- Stochastic Planning and Lifted Inference
- Simulation-Based Inference for Global Health Decisions
- Towards Verified Stochastic Variational Inference for Probabilistic Programs