Probabilistic Programs with Stochastic Conditioning

Abstract

We tackle the problem of conditioning probabilistic programs on distributions of observable variables. Probabilistic programs are usually conditioned on samples from the joint data distribution, which we refer to as deterministic conditioning. However, in many real-life scenarios, the observations are given as marginal distributions, summary statistics, or samplers. Conventional probabilistic programming systems lack adequate means for modeling and inference in such scenarios. We propose a generalization of deterministic conditioning to stochastic conditioning, that is, conditioning on the marginal distribution of a variable taking a particular form. To this end, we first define the formal notion of stochastic conditioning and discuss its key properties. We then show how to perform inference in the presence of stochastic conditioning. We demonstrate potential usage of stochastic conditioning on several case studies which involve various kinds of stochastic conditioning and are difficult to solve otherwise. Although we present stochastic conditioning in the context of probabilistic programming, our formalization is general and applicable to other settings.

References in corpus (15)

• The No-U-Turn Sampler: Adaptively Setting Path Lengths in Hamiltonian Monte Carlo
• The pseudo-marginal approach for efficient Monte Carlo computations
• Church: a language for generative models
• Stochastic Gradient Hamiltonian Monte Carlo
• Austerity in MCMC Land: Cutting the Metropolis-Hastings Budget
• Speeding Up MCMC by Efficient Data Subsampling
• Semantics for probabilistic programming: higher-order functions, continuous distributions, and soft constraints
• The Penalty Method for Random Walks with Uncertain Energies
• Deep Probabilistic Programming
• Firefly Monte Carlo: Exact MCMC with Subsets of Data
• On Nesting Monte Carlo Estimators
• Coupled MCMC with a randomized acceptance probability
• Black-Box Policy Search with Probabilistic Programs
• Nested Reasoning About Autonomous Agents Using Probabilistic Programs
• The Random Conditional Distribution for Higher-Order Probabilistic Inference