5 papers
Gumbel Straight Flow: Distilling Autoregressive Models into One-step Flow Maps
Yeongmin Kim, Arnaud Doucet, Andrew Campbell +3
We present Gumbel Straight Flow (GSF), a continuous flow map language model that leverages the noise-data coupling of a pretrained autoregressive language (AR) model. We theoretica…
DiffusionGemma Technical Report
DiffusionGemma Team, Adrien Ali Taïga, James Assiene +41
We introduce DiffusionGemma, an experimental open-weight language model that uses discrete diffusion to generate text at exceptionally high speed. Rather than decoding one token at…
Dual-Rate Diffusion: Accelerating diffusion models with an interleaved heavy-light network
Grigory Bartosh, David Ruhe, Emiel Hoogeboom +3
Diffusion models achieve state-of-the-art generative performance but suffer from high computational costs during inference due to the repeated evaluation of a heavy neural network.…
Beyond Single Tokens: Distilling Discrete Diffusion Models via Discrete MMD
Emiel Hoogeboom, David Ruhe, Jonathan Heek +2
It is currently difficult to distill discrete diffusion models. In contrast, continuous diffusion literature has many distillation approaches methods that can reduce sampling steps…
Clifford Group Equivariant Diffusion Models for 3D Molecular Generation
Cong Liu, Sharvaree Vadgama, David Ruhe +2
This paper explores leveraging the Clifford algebra's expressive power for $\E(n)$-equivariant diffusion models. We utilize the geometric products between Clifford multivectors and…