7 papers
Introduction to Stochastic Differential Equations for Generative Machine Learning: A Variational Perspective
Ole Winther, Paul Jeha, Sander Dieleman +3
The use of ordinary and stochastic differential equations has led to substantial progress in generative machine learning with applications to, for example, image, video and biomole…
Mix, Don't Tune: Bilingual Pre-Training Outperforms Hyperparameter Search in Data-Constrained Settings
Paul Jeha, Anastasiia Sedova, Louis Béthune +4
For most languages of the world, language model pre-training operates in a data-constrained regime where models must repeat their training data many times, degrading generalization…
CREPE: Controlling Diffusion with Replica Exchange
Jiajun He, Paul Jeha, Peter Potaptchik +5
Inference-time control of diffusion models aims to steer model outputs to satisfy new constraints without retraining. Previous approaches have mostly relied on heuristic guidance o…
Debiasing Guidance for Discrete Diffusion with Sequential Monte Carlo
Cheuk Kit Lee, Paul Jeha, Jes Frellsen +3
Discrete diffusion models are a class of generative models that produce samples from an approximated data distribution within a discrete state space. Often, there is a need to targ…
Learning Energy-Based Models by Self-normalising the Likelihood
Hugo Senetaire, Paul Jeha, Pierre-Alexandre Mattei +1
Training an energy-based model (EBM) with maximum likelihood is challenging due to the intractable normalisation constant. Traditional methods rely on expensive Markov chain Monte…
Generative Diffusion Models for Sequential Recommendations
Sharare Zolghadr, Ole Winther, Paul Jeha
Generative models such as Variational Autoencoders (VAEs) and Generative Adversarial Networks (GANs) have shown promise in sequential recommendation tasks. However, they face chall…