Functional Tensors for Probabilistic Programming
arXiv:1910.10775
Abstract
It is a significant challenge to design probabilistic programming systems that can accommodate a wide variety of inference strategies within a unified framework. Noting that the versatility of modern automatic differentiation frameworks is based in large part on the unifying concept of tensors, we describe a software abstraction for integration --functional tensors-- that captures many of the benefits of tensors, while also being able to describe continuous probability distributions. Moreover, functional tensors are a natural candidate for generalized variable elimination and parallel-scan filtering algorithms that enable parallel exact inference for a large family of tractable modeling motifs. We demonstrate the versatility of functional tensors by integrating them into the modeling frontend and inference backend of the Pyro programming language. In experiments we show that the resulting framework enables a large variety of inference strategies, including those that mix exact and approximate inference.
arXiv commentSubmitted to ICML 2020
References in corpus (9)
- Adam: A Method for Stochastic Optimization
- Stochastic Backpropagation and Approximate Inference in Deep Generative Models
- Expectation Propagation for approximate Bayesian inference
- Pyro: Deep Universal Probabilistic Programming
- momentuHMM: R package for generalized hidden Markov models of animal movement
- Gradient Estimation Using Stochastic Computation Graphs
- DiCE: The Infinitely Differentiable Monte-Carlo Estimator
- Tensor Variable Elimination for Plated Factor Graphs
- Autoconj: Recognizing and Exploiting Conjugacy Without a Domain-Specific Language
Cited by in corpus (6)
- Tropical Tensor Network for Ground States of Spin Glasses
- Nonlinear System Identification: Learning while respecting physical models using a sequential Monte Carlo method
- Probabilistic Programming with Programmable Variational Inference
- Learning Proposals for Probabilistic Programs with Inference Combinators
- Adversarial training applied to Convolutional Neural Network for photometric redshift predictions
- Contributions to Large Scale Bayesian Inference and Adversarial Machine Learning