38 citations · 86 across the 19 of their papers we have counts for
19 papers
GradInf: Gradient Estimation as Probabilistic Inference
Gaurav Arya, Mathieu Huot, Moritz Schauer +2
Gradient estimation -- the task of computing the gradient of the expected value of a probabilistic program -- has diverse applications in scientific computing, but is notoriously d…
Incremental Computation for Efficient Programmable Inference in Probabilistic Programs
Fabian Zaiser, Jack Czenszak, Martin C. Rinard +2
Inference in probabilistic programs generally requires evaluating many possible program executions to find those of high posterior density. To scale inference to large datasets, it…
Modeling Open-World Cognition as On-Demand Synthesis of Probabilistic Models
Lionel Wong, Katherine M. Collins, Lance Ying +8
When faced with novel situations, people are able to marshal relevant considerations from a wide range of background knowledge and put these to use in inferences and predictions. W…
Syntactic and Semantic Control of Large Language Models via Sequential Monte Carlo
João Loula, Benjamin LeBrun, Li Du +12
A wide range of LM applications require generating text that conforms to syntactic or semantic constraints. Imposing such constraints can be naturally framed as probabilistic condi…
Self-Steering Language Models
Gabriel Grand, Joshua B. Tenenbaum, Vikash K. Mansinghka +2
While test-time reasoning enables language models (LMs) to tackle complex tasks, searching or planning in natural language can be slow, costly, and error-prone. But even when LMs s…
Fast Controlled Generation from Language Models with Adaptive Weighted Rejection Sampling
Benjamin Lipkin, Benjamin LeBrun, Jacob Hoover Vigly +9
The dominant approach to generating from language models subject to some constraint is locally constrained decoding (LCD), incrementally sampling tokens at each time step such that…
Probabilistic Programming with Programmable Variational Inference
McCoy R. Becker, Alexander K. Lew, Xiaoyan Wang +4
Compared to the wide array of advanced Monte Carlo methods supported by modern probabilistic programming languages (PPLs), PPL support for variational inference (VI) is less develo…
GenSQL: A Probabilistic Programming System for Querying Generative Models of Database Tables
Mathieu Huot, Matin Ghavami, Alexander K. Lew +6
This article presents GenSQL, a probabilistic programming system for querying probabilistic generative models of database tables. By augmenting SQL with only a few key primitives f…
Differentiating Metropolis-Hastings to Optimize Intractable Densities
Gaurav Arya, Ruben Seyer, Frank Schäfer +7
We develop an algorithm for automatic differentiation of Metropolis-Hastings samplers, allowing us to differentiate through probabilistic inference, even if the model has discrete…
From Word Models to World Models: Translating from Natural Language to the Probabilistic Language of Thought
Lionel Wong, Gabriel Grand, Alexander K. Lew +4
How does language inform our downstream thinking? In particular, how do humans make meaning from language--and how can we leverage a theory of linguistic meaning to build machines…
Sequential Monte Carlo Steering of Large Language Models using Probabilistic Programs
Alexander K. Lew, Tan Zhi-Xuan, Gabriel Grand +1
Even after fine-tuning and reinforcement learning, large language models (LLMs) can be difficult, if not impossible, to control reliably with prompts alone. We propose a new infere…
PAP Spaces: Reasoning Denotationally About Higher-Order, Recursive Probabilistic and Differentiable Programs
Mathieu Huot, Alexander K. Lew, Vikash K. Mansinghka +1
We introduce a new setting, the category of PAP spaces, for reasoning denotationally about expressive differentiable and probabilistic programming languages. Our semantics is ge…
ADEV: Sound Automatic Differentiation of Expected Values of Probabilistic Programs
Alexander K. Lew, Mathieu Huot, Sam Staton +1
Optimizing the expected values of probabilistic processes is a central problem in computer science and its applications, arising in fields ranging from artificial intelligence to o…
Recursive Monte Carlo and Variational Inference with Auxiliary Variables
Alexander K. Lew, Marco Cusumano-Towner, Vikash K. Mansinghka
A key design constraint when implementing Monte Carlo and variational inference algorithms is that it must be possible to cheaply and exactly evaluate the marginal densities of pro…
Towards Denotational Semantics of AD for Higher-Order, Recursive, Probabilistic Languages
Alexander K. Lew, Mathieu Huot, Vikash K. Mansinghka
Automatic differentiation (AD) aims to compute derivatives of user-defined functions, but in Turing-complete languages, this simple specification does not fully capture AD's behavi…
Automating Involutive MCMC using Probabilistic and Differentiable Programming
Marco Cusumano-Towner, Alexander K. Lew, Vikash K. Mansinghka
Involutive MCMC is a unifying mathematical construction for MCMC kernels that generalizes many classic and state-of-the-art MCMC algorithms, from reversible jump MCMC to kernels ba…
PClean: Bayesian Data Cleaning at Scale with Domain-Specific Probabilistic Programming
Alexander K. Lew, Monica Agrawal, David Sontag +1
Data cleaning is naturally framed as probabilistic inference in a generative model of ground-truth data and likely errors, but the diversity of real-world error patterns and the ha…
Bayesian causal inference via probabilistic program synthesis
Sam Witty, Alexander Lew, David Jensen +1
Causal inference can be formalized as Bayesian inference that combines a prior distribution over causal models and likelihoods that account for both observations and interventions.…
Few-Shot Bayesian Imitation Learning with Logical Program Policies
Tom Silver, Kelsey R. Allen, Alex K. Lew +2
Humans can learn many novel tasks from a very small number (1--5) of demonstrations, in stark contrast to the data requirements of nearly tabula rasa deep learning methods. We prop…