Church: a language for generative models
arXiv:1206.3255
Abstract
We introduce Church, a universal language for describing stochastic generative processes. Church is based on the Lisp model of lambda calculus, containing a pure Lisp as its deterministic subset. The semantics of Church is defined in terms of evaluation histories and conditional distributions on such histories. Church also includes a novel language construct, the stochastic memoizer, which enables simple description of many complex non-parametric models. We illustrate language features through several examples, including: a generalized Bayes net in which parameters cluster over trials, infinite PCFGs, planning by inference, and various non-parametric clustering models. Finally, we show how to implement query on any Church program, exactly and approximately, using Monte Carlo techniques.
arXiv commentMinor revisions. Fixed errors in author list
Cited by in corpus (107)
- Pyro: Deep Universal Probabilistic Programming
- Automatic Differentiation Variational Inference
- A Bayesian Sampling Approach to Exploration in Reinforcement Learning
- Hinge-Loss Markov Random Fields and Probabilistic Soft Logic
- Semantics for probabilistic programming: higher-order functions, continuous distributions, and soft constraints
- A Convenient Category for Higher-Order Probability Theory
- Scaling Exact Inference for Discrete Probabilistic Programs
- TerpreT: A Probabilistic Programming Language for Program Induction
- Automated Variational Inference in Probabilistic Programming
- Denotational validation of higher-order Bayesian inference
- Approximate Bayesian Image Interpretation using Generative Probabilistic Graphics Programs
- Deep Amortized Inference for Probabilistic Programs
- SPPL: Probabilistic Programming with Fast Exact Symbolic Inference
- Programs as Black-Box Explanations
- On the computability of conditional probability
- Measure Transformer Semantics for Bayesian Machine Learning
- Formal verification of higher-order probabilistic programs
- Nonlinear System Identification: Learning while respecting physical models using a sequential Monte Carlo method
- Computable de Finetti measures
- A Dynamic Programming Algorithm for Inference in Recursive Probabilistic Programs
- C3: Lightweight Incrementalized MCMC for Probabilistic Programs using Continuations and Callsite Caching
- Black-Box Policy Search with Probabilistic Programs
- Inducing Probabilistic Programs by Bayesian Program Merging
- Monolingual Probabilistic Programming Using Generalized Coroutines
- Particle Gibbs with Ancestor Sampling for Probabilistic Programs
- Accelerating Inference: towards a full Language, Compiler and Hardware stack
- Compiling Universal Probabilistic Programming Languages with Efficient Parallel Sequential Monte Carlo Inference
- Gibbs Sampling in Open-Universe Stochastic Languages
- Swift: Compiled Inference for Probabilistic Programming Languages
- Driving Markov chain Monte Carlo with a dependent random stream
- Traceability of Deep Neural Networks
- Reparameterization Gradient for Non-differentiable Models
- Towards common-sense reasoning via conditional simulation: legacies of Turing in Artificial Intelligence
- A Credit Assignment Compiler for Joint Prediction
- Probabilistic Programming Concepts
- Paradoxes of Probabilistic Programming
- Learning Probabilistic Programs
- Nonparametric Bayesian Density Modeling with Gaussian Processes
- Simplifying Dependent Reductions in the Polyhedral Model
- Divide, Conquer, and Combine: a New Inference Strategy for Probabilistic Programs with Stochastic Support
- Learning Proposals for Probabilistic Programs with Inference Combinators
- Distribution Bisimilarity via the Power of Convex Algebras
- On the Pitfalls of Nested Monte Carlo
- Nested Reasoning About Autonomous Agents Using Probabilistic Programs
- Automating Involutive MCMC using Probabilistic and Differentiable Programming
- Amortized Rejection Sampling in Universal Probabilistic Programming
- A Probabilistic Dependent Type System based on Non-Deterministic Beta Reduction
- Beginner's Luck: A Language for Property-Based Generators
- Nesting Probabilistic Programs
- Probabilistic Neural Programs
- Higher-Order Bayesian Networks, Exactly (Extended version)
- An Application of Computable Distributions to the Semantics of Probabilistic Programs
- InferSpark: Statistical Inference at Scale
- On the computability of graphons
- Bayesian Optimization for Probabilistic Programs
- Compiling Relational Database Schemata into Probabilistic Graphical Models
- The Random Conditional Distribution for Higher-Order Probabilistic Inference
- Deriving Probability Density Functions from Probabilistic Functional Programs
- Cost Analysis of Nondeterministic Probabilistic Programs
- Soft Constraints for Inference with Declarative Knowledge
- Sublinear-Time Approximate MCMC Transitions for Probabilistic Programs
- Hamiltonian Monte Carlo for Probabilistic Programs with Discontinuities
- Unifying AI Algorithms with Probabilistic Programming using Implicitly Defined Representations
- Maximum a Posteriori Estimation by Search in Probabilistic Programs
- Probabilistic programming for birth-death models of evolution using an alive particle filter with delayed sampling
- DynamicPPL: Stan-like Speed for Dynamic Probabilistic Models
- Evaluating probabilistic programming languages for simulating quantum correlations
- Efficient Probabilistic Inference in the Quest for Physics Beyond the Standard Model
- Applications of Probabilistic Programming (Master's thesis, 2015)
- Probabilistic Program Abstractions
- PClean: Bayesian Data Cleaning at Scale with Domain-Specific Probabilistic Programming
- Spreadsheet Probabilistic Programming
- RankPL: A Qualitative Probabilistic Programming Language
- SIMPL: A DSL for Automatic Specialization of Inference Algorithms
- Metric Reasoning about -Terms: the Affine Case (Long Version)
- Static Analysis for Probabilistic Programs
- From high-level inference algorithms to efficient code
- Encapsulating models and approximate inference programs in probabilistic modules
- Slice Sampling for Probabilistic Programming
- Probabilistic Termination by Monadic Affine Sized Typing (Long Version)
- Joint Distributions in Probabilistic Semantics
- Coarse-to-Fine Sequential Monte Carlo for Probabilistic Programs
- Herded Gibbs Sampling
- On Open-Universe Causal Reasoning
- A model of stochastic memoization and name generation in probabilistic programming: categorical semantics via monads on presheaf categories
- On Higher-Order Probabilistic Subrecursion
- Programming with Neural Surrogates of Programs
- Contextual Equivalence for a Probabilistic Language with Continuous Random Variables and Recursion
- Running Probabilistic Programs Backwards
- Effectful Programming in Declarative Languages with an Emphasis on Non-Determinism: Applications and Formal Reasoning
- Approximations in Probabilistic Programs
- Meta-Learning an Inference Algorithm for Probabilistic Programs
- Stochastic Probabilistic Programs
- Probabilistic Programs with Stochastic Conditioning
- Proving Almost-Sure Termination of Probabilistic Programs via Incremental Pruning
- Stochastically Differentiable Probabilistic Programs
- Whittemore: An embedded domain specific language for causal programming
- Towards Verified Stochastic Variational Inference for Probabilistic Programs
- Doubly Bayesian Optimization
- Compositional Inference Metaprogramming with Convergence Guarantees
- Abstractions for AI-Based User Interfaces and Systems
- Semiring Programming: A Declarative Framework for Generalized Sum Product Problems
- Probabilistic Safety Programs
- Lazy Factored Inference for Functional Probabilistic Programming
- Path Finding under Uncertainty through Probabilistic Inference
- A Useful Algebraic System of Statistical Models
- Probabilities on Sentences in an Expressive Logic