Black-Box Policy Search with Probabilistic Programs

Abstract

In this work, we explore how probabilistic programs can be used to represent policies in sequential decision problems. In this formulation, a probabilistic program is a black-box stochastic simulator for both the problem domain and the agent. We relate classic policy gradient techniques to recently introduced black-box variational methods which generalize to probabilistic program inference. We present case studies in the Canadian traveler problem, Rock Sample, and a benchmark for optimal diagnosis inspired by Guess Who. Each study illustrates how programs can efficiently represent policies using moderate numbers of parameters.

References in corpus (5)

• Multiple Object Recognition with Visual Attention
• Church: a language for generative models
• Automated Variational Inference in Probabilistic Programming
• Selecting Computations: Theory and Applications
• New inference strategies for solving Markov Decision Processes using reversible jump MCMC

Cited by in corpus (10)

• Quantum synchronization and quantum state sharing in irregular complex network
• The Generalized Reparameterization Gradient
• Overdispersed Black-Box Variational Inference
• Nested Reasoning About Autonomous Agents Using Probabilistic Programs
• Bayesian Optimization for Probabilistic Programs
• Probabilistic Programs with Stochastic Conditioning
• Bayesian Policy Search for Stochastic Domains
• Stochastic Planning and Lifted Inference
• Simulation-Based Inference for Global Health Decisions
• Towards Verified Stochastic Variational Inference for Probabilistic Programs