113 citations · 491 across the 32 of their papers we have counts for
34 papers
Smoothness Analysis for Probabilistic Programs with Application to Optimised Variational Inference
Wonyeol Lee, Xavier Rival, Hongseok Yang
We present a static analysis for discovering differentiable or more generally smooth parts of a given probabilistic program, and show how the analysis can be used to improve the pa…
Learning Symmetric Rules with SATNet
Sangho Lim, Eun-Gyeol Oh, Hongseok Yang
SATNet is a differentiable constraint solver with a custom backpropagation algorithm, which can be used as a layer in a deep-learning system. It is a promising proposal for bridgin…
Scale Mixtures of Neural Network Gaussian Processes
Hyungi Lee, Eunggu Yun, Hongseok Yang +1
Recent works have revealed that infinitely-wide feed-forward or recurrent neural networks of any architecture correspond to Gaussian processes referred to as Neural Network Gaussia…
-Stable convergence of heavy-tailed infinitely-wide neural networks
Paul Jung, Hoil Lee, Jiho Lee +1
We consider infinitely-wide multi-layer perceptrons (MLPs) which are limits of standard deep feed-forward neural networks. We assume that, for each layer, the weights of an MLP are…
Meta-Learning an Inference Algorithm for Probabilistic Programs
Gwonsoo Che, Hongseok Yang
We present a meta-algorithm for learning a posterior-inference algorithm for restricted probabilistic programs. Our meta-algorithm takes a training set of probabilistic programs th…
Bayesian Policy Search for Stochastic Domains
David Tolpin, Yuan Zhou, Hongseok Yang
AI planning can be cast as inference in probabilistic models, and probabilistic programming was shown to be capable of policy search in partially observable domains. Prior work int…
Probabilistic Programs with Stochastic Conditioning
David Tolpin, Yuan Zhou, Tom Rainforth +1
We tackle the problem of conditioning probabilistic programs on distributions of observable variables. Probabilistic programs are usually conditioned on samples from the joint data…
On Correctness of Automatic Differentiation for Non-Differentiable Functions
Wonyeol Lee, Hangyeol Yu, Xavier Rival +1
Differentiation lies at the core of many machine-learning algorithms, and is well-supported by popular autodiff systems, such as TensorFlow and PyTorch. Originally, these systems h…
Stochastically Differentiable Probabilistic Programs
David Tolpin, Yuan Zhou, Hongseok Yang
Probabilistic programs with mixed support (both continuous and discrete latent random variables) commonly appear in many probabilistic programming systems (PPSs). However, the exis…
Differentiable Algorithm for Marginalising Changepoints
Hyoungjin Lim, Gwonsoo Che, Wonyeol Lee +1
We present an algorithm for marginalising changepoints in time-series models that assume a fixed number of unknown changepoints. Our algorithm is differentiable with respect to its…
Divide, Conquer, and Combine: a New Inference Strategy for Probabilistic Programs with Stochastic Support
Yuan Zhou, Hongseok Yang, Yee Whye Teh +1
Universal probabilistic programming systems (PPSs) provide a powerful framework for specifying rich probabilistic models. They further attempt to automate the process of drawing in…
Towards Verified Stochastic Variational Inference for Probabilistic Programs
Wonyeol Lee, Hangyeol Yu, Xavier Rival +1
Probabilistic programming is the idea of writing models from statistics and machine learning using program notations and reasoning about these models using generic inference engine…
LF-PPL: A Low-Level First Order Probabilistic Programming Language for Non-Differentiable Models
Yuan Zhou, Bradley J. Gram-Hansen, Tobias Kohn +3
We develop a new Low-level, First-order Probabilistic Programming Language (LF-PPL) suited for models containing a mix of continuous, discrete, and/or piecewise-continuous variable…
A Generalization of Hierarchical Exchangeability on Trees to Directed Acyclic Graphs
Paul Jung, Jiho Lee, Sam Staton +1
Motivated by the problem of designing inference-friendly Bayesian nonparametric models in probabilistic programming languages, we introduce a general class of partially exchangeabl…
An Introduction to Probabilistic Programming
Jan-Willem van de Meent, Brooks Paige, Hongseok Yang +1
This book is a graduate-level introduction to probabilistic programming. It not only provides a thorough background for anyone wishing to use a probabilistic programming system, bu…
Inference Trees: Adaptive Inference with Exploration
Tom Rainforth, Yuan Zhou, Xiaoyu Lu +4
We introduce inference trees (ITs), a new class of inference methods that build on ideas from Monte Carlo tree search to perform adaptive sampling in a manner that balances explora…
Reparameterization Gradient for Non-differentiable Models
Wonyeol Lee, Hangyeol Yu, Hongseok Yang
We present a new algorithm for stochastic variational inference that targets at models with non-differentiable densities. One of the key challenges in stochastic variational infere…
Hamiltonian Monte Carlo for Probabilistic Programs with Discontinuities
Bradley Gram-Hansen, Yuan Zhou, Tobias Kohn +3
Hamiltonian Monte Carlo (HMC) is arguably the dominant statistical inference algorithm used in most popular "first-order differentiable" Probabilistic Programming Languages (PPLs).…
The Beta-Bernoulli process and algebraic effects
Sam Staton, Dario Stein, Hongseok Yang +3
In this paper we use the framework of algebraic effects from programming language theory to analyze the Beta-Bernoulli process, a standard building block in Bayesian models. Our an…
Denotational validation of higher-order Bayesian inference
Adam Ścibior, Ohad Kammar, Matthijs Vákár +7
We present a modular semantic account of Bayesian inference algorithms for probabilistic programming languages, as used in data science and machine learning. Sophisticated inferenc…
On Nesting Monte Carlo Estimators
Tom Rainforth, Robert Cornish, Hongseok Yang +2
Many problems in machine learning and statistics involve nested expectations and thus do not permit conventional Monte Carlo (MC) estimation. For such problems, one must nest estim…
Algebraic Laws for Weak Consistency (Extended Version)
Andrea Cerone, Alexey Gotsman, Hongseok Yang
Modern distributed systems often rely on so called weakly-consistent databases, which achieve scalability by sacrificing the consistency guarantee of distributed transaction proces…
A Convenient Category for Higher-Order Probability Theory
Chris Heunen, Ohad Kammar, Sam Staton +1
Higher-order probabilistic programming languages allow programmers to write sophisticated models in machine learning and statistics in a succinct and structured way, but step outsi…
On the Pitfalls of Nested Monte Carlo
Tom Rainforth, Robert Cornish, Hongseok Yang +1
There is an increasing interest in estimating expectations outside of the classical inference framework, such as for models expressed as probabilistic programs. Many of these conte…