8 papers
When are likely answers right? On Sequence Probability and Correctness in LLMs
Johannes Zenn, Jonas Geiping
Many decoding methods for large language models can be understood as shifting probability mass toward outputs that are more likely under the model, either locally at the token leve…
Efficient Test-Time Inference via Deterministic Exploration of Truncated Decoding Trees
Xueyan Li, Johannes Zenn, Ekaterina Fadeeva +3
Self-consistency boosts inference-time performance by sampling multiple reasoning traces in parallel and voting. However, in constrained domains like math and code, this strategy i…
Language Models For Generalised PDDL Planning: Synthesising Sound and Programmatic Policies
Dillon Z. Chen, Johannes Zenn, Tristan Cinquin +1
We study the usage of language models (LMs) for planning over world models specified in the Planning Domain Definition Language (PDDL). We prompt LMs to generate Python programs th…
Flipping Against All Odds: Reducing LLM Coin Flip Bias via Verbalized Rejection Sampling
Tim Z. Xiao, Johannes Zenn, Zhen Liu +3
Large language models (LLMs) can often accurately describe probability distributions using natural language, yet they still struggle to generate faithful samples from them. This mi…
Balancing Molecular Information and Empirical Data in the Prediction of Physico-Chemical Properties
Johannes Zenn, Dominik Gond, Fabian Jirasek +1
Predicting the physico-chemical properties of pure substances and mixtures is a central task in thermodynamics. Established prediction methods range from fully physics-based ab-ini…
Differentiable Annealed Importance Sampling Minimizes The Symmetrized Kullback-Leibler Divergence Between Initial and Target Distribution
Johannes Zenn, Robert Bamler
Differentiable annealed importance sampling (DAIS), proposed by Geffner & Domke (2021) and Zhang et al. (2021), allows optimizing over the initial distribution of AIS. In this pape…