6 papers
One-Step Generation via Riemannian Wasserstein Gradient Flows
David Li, Chanhyuk Lee, Jaehoon Yoo +7
Recently, Drifting Models and Wasserstein Gradient Flows have attracted substantial attention because they move iterative distributional refinement to training and amortize it into…
IDRF: Inverse-Distilled Reward Fine-tuning of Masked Discrete Diffusion Models
Vladislav Gromadskii, David Li, Samson Gourevitch +4
Masked discrete diffusion models offer a promising alternative to autoregressive generation, but iterative sampling can be costly, and intractable sequence likelihoods complicate r…
Alpha Diffusion Language Models: Factorization Alone Is Not the Problem
Nikita Gushchin, Dmitry Baranchuk, Alexander Korotin
Discrete diffusion language models can generate multiple tokens in parallel, but reducing the number of denoising steps can lead to inconsistent predictions. Standard cross-entropy…
E-MoE: Enhanced Mixture-of-Experts for Non-Factorized Diffusion Language Models
Arseny Ivanov, Alexander Kolesov, Alexander Korotin +2
Masked diffusion models (MDMs) generate sequences by progressively unmasking several tokens per denoising step, but their reverse process is typically factorized over positions, li…
TUBE: Tangent Upper Bound on Evidence for Discrete Diffusion Language Models
Arseny Ivanov, Sergei Kholkin, Vladislav Gromadskii +3
Log-likelihood is a standard metric for evaluating generative models. Unfortunately, in contrast to autoregressive models (ARMs), discrete diffusion models generally do not admit e…
How to Train Your Latent Diffusion Language Model Jointly With the Latent Space
Viacheslav Meshchaninov, Alexander Shabalin, Egor Chimbulatov +4
Latent diffusion models offer an attractive alternative to discrete diffusion for non-autoregressive text generation by operating on continuous text representations and denoising e…