6 papers
Elicitation without Backpropagation: Steering Model Behavior by Optimizing the Latent Posterior
Garrett Baker, Vinayak Pathak, Daniel Murfet +1
In the \emph{latent posterior model} of transformer behavior, the next-token distribution arises from a posterior over latent predictive models conditioned on the context, mixed to…
What does a Bayes-filtered transformer believe? A predictive Monte Carlo approach
Afiq Abdillah Effiezal Aswadi, Haotong Ma, Susan Wei
A Bayes-filtered transformer (BFT) is a transformer trained on sequences that are generated in two steps: first a latent task is drawn from a prior, then observations are drawn con…
TabMGP: Martingale Posterior with TabPFN
Kenyon Ng, Edwin Fong, David T. Frazier +2
Bayesian inference provides principled uncertainty quantification but is often limited by the challenges of prior and likelihood elicitation. The martingale posterior (MGP) (Fong e…
Uncertainty Decomposition for Bayes-Filtered Transformers via Bayesian Predictive Inference
Sandra Fortini, Kenyon Ng, Sonia Petrone +2
Bayes-filtered transformers are transformers meta-learned on sequences from a prior predictive distribution to approximate the corresponding posterior predictive distribution. They…
Temperature Optimization for Bayesian Deep Learning
Kenyon Ng, Chris van der Heide, Liam Hodgkinson +1
The Cold Posterior Effect (CPE) is a phenomenon in Bayesian Deep Learning (BDL), where tempering the posterior to a cold temperature often improves the predictive performance of th…
Pathwise Gradient Variance Reduction with Control Variates in Variational Inference
Kenyon Ng, Susan Wei
Variational inference in Bayesian deep learning often involves computing the gradient of an expectation that lacks a closed-form solution. In these cases, pathwise and score-functi…