27 papers
Lightweight Latent Reasoning for Narrative Tasks
Alexander Gurung, Esmeralda S. Whitammer, Mirella Lapata
Large language models (LLMs) tackle complex tasks by generating long chains of thought or "reasoning traces" that act as latent variables in the generation of an output given a que…
Reinforced sequential Monte Carlo for amortised sampling
Sanghyeok Choi, Sarthak Mittal, VÃctor Elvira +2
This paper proposes a synergy of amortised and particle-based methods for sampling from distributions defined by unnormalised density functions. We state a connection between seque…
Discrete diffusion samplers and bridges: Off-policy algorithms and applications in latent spaces
Arran Carter, Sanghyeok Choi, Kirill Tamogashev +2
Sampling from a distribution known up to a normalising constant is an important and challenging problem in statistics. Recent years have seen the…
How to Approximate Inference with Subtractive Mixture Models
Lena Zellinger, Nicola Branchini, Lennert De Smet +3
Classical mixture models (MMs) are widely used tractable proposals for approximate inference settings such as variational inference (VI) and importance sampling (IS). Recently, mix…
Likelihood Hacking in Probabilistic Program Synthesis
Jacek Karwowski, Younesse Kaddar, Zihuiwen Ye +3
When language models are trained by reinforcement learning (RL) to write probabilistic programs, they can artificially inflate their marginal-likelihood reward by producing program…
A Comedy of Estimators: On KL Regularization in RL Training of LLMs
Vedant Shah, Johan Obando-Ceron, Vineet Jain +10
The reasoning performance of large language models (LLMs) can be substantially improved by training them with reinforcement learning (RL). The RL objective for LLM training involve…