7 papers · 1 filter
Soft Specialists: -Rényi Ensembles for Uncertainty-Aware LLM Post-Training
Paula Cordero-Encinar, Georgy Tyukin, Andrew B. Duncan
Existing training approaches for large language models learn a single set of parameters, based on large volumes of data, which is typically heterogeneous, conflicting and often out…
Diffusion Path Samplers via Sequential Monte Carlo
James Matthew Young, Paula Cordero-Encinar, Sebastian Reich +2
We develop diffusion-based samplers for target distributions known up to a normalising constant. To this end, we rely on the well-known diffusion path that smoothly interpolates be…
Certified Self-Consistency: Statistical Guarantees and Test-Time Training for Reliable Reasoning in LLMs
Paula Cordero-Encinar, Andrew B. Duncan
Recent advances such as self-consistency and test-time reinforcement learning (TTRL) improve the reliability of large language models (LLMs) without additional supervision, yet the…
Uniform-in-time convergence bounds for Persistent Contrastive Divergence algorithms
Paul Felix Valsecchi Oliva, O. Deniz Akyildiz, Andrew Duncan
We propose a continuous-time formulation of a noisy persistent contrastive divergence (PCD)-like method for maximum likelihood estimation (MLE) of unnormalised densities. Our appro…
Non-asymptotic Analysis of Diffusion Annealed Langevin Monte Carlo for Generative Modelling
Paula Cordero-Encinar, O. Deniz Akyildiz, Andrew B. Duncan
We investigate the theoretical properties of general diffusion (interpolation) paths and their Langevin Monte Carlo implementation, referred to as diffusion annealed Langevin Monte…
Energy-Based Modelling for Discrete and Mixed Data via Heat Equations on Structured Spaces
Tobias Schröder, Zijing Ou, Yingzhen Li +1
Energy-based models (EBMs) offer a flexible framework for probabilistic modelling across various data domains. However, training EBMs on data in discrete or mixed state spaces pose…
Deep Optimal Sensor Placement for Black Box Stochastic Simulations
Paula Cordero-Encinar, Tobias Schröder, Peter Yatsyshin +1
Selecting cost-effective optimal sensor configurations for subsequent inference of parameters in black-box stochastic systems faces significant computational barriers. We propose a…