10 papers
BlockGen: Flexible Blockwise Sequence Modeling with Hybrid Samplers
Justin Deschenaux, Caglar Gulcehre
Is the uniform-state diffusion framework a more powerful paradigm for discrete diffusion? Recent studies indicate that this may be the case. In combination with predictor-corrector…
Language Modeling with Hyperspherical Flows
Justin Deschenaux, Caglar Gulcehre
Discrete Diffusion Language Models progressed rapidly as an alternative to autoregressive (AR) models, motivated by their parallel generation abilities. However, for tractability,…
Fixed-Point Masked Generative Modeling
Andrea Miele, Yiming Qin, Alba Carballo-Castro +2
Masked Generative Models (MGMs) enable parallel decoding and achieve strong performance across modalities, but require full-sequence bidirectional transformers at every step, makin…
The Diffusion Duality, Chapter II: -Samplers
Justin Deschenaux, Caglar Gulcehre, Subham Sekhar Sahoo
Uniform-state discrete diffusion models excel at few-step generation and guidance due to their ability to self-correct, making them preferred over autoregressive or Masked diffusio…
Loopholing Discrete Diffusion: Deterministic Bypass of the Sampling Wall
Mingyu Jo, Jaesik Yoon, Justin Deschenaux +2
Discrete diffusion models offer a promising alternative to autoregressive generation through parallel decoding, but they suffer from a sampling wall: once categorical sampling occu…
Partition Generative Modeling: Masked Modeling Without Masks
Justin Deschenaux, Lan Tran, Caglar Gulcehre
Masked generative models (MGMs) can generate tokens in parallel and in any order, unlike autoregressive models (ARMs), which decode one token at a time, left-to-right. However, MGM…