6 papers
Unifying Masked Diffusion Models with Various Generation Orders and Beyond
Chunsan Hong, Sanghyun Lee, Jong Chul Ye
Masked diffusion models (MDMs) are a potential alternative to autoregressive models (ARMs) for language generation, but generation quality depends critically on the generation orde…
Understanding and Accelerating the Training of Masked Diffusion Language Models
Chunsan Hong, Sanghyun Lee, Chieh-Hsin Lai +5
Masked diffusion models (MDMs) have emerged as a promising alternative to autoregressive models (ARMs) for language modeling. However, MDMs are known to learn substantially more sl…
Lookahead Unmasking Elicits Accurate Decoding in Diffusion Language Models
Sanghyun Lee, Seungryong Kim, Jongho Park +1
Masked Diffusion Models (MDMs) as language models generate by iteratively unmasking tokens, yet their performance crucially depends on the inference time order of unmasking. Prevai…
Effective Test-Time Scaling of Discrete Diffusion through Iterative Refinement
Sanghyun Lee, Sunwoo Kim, Seungryong Kim +2
Test-time scaling through reward-guided generation remains largely unexplored for discrete diffusion models despite its potential as a promising alternative. In this work, we intro…
Where and How to Perturb: On the Design of Perturbation Guidance in Diffusion and Flow Models
Donghoon Ahn, Jiwon Kang, Sanghyun Lee +7
Recent guidance methods in diffusion models steer reverse sampling by perturbing the model to construct an implicit weak model and guide generation away from it. Among these approa…
A Noise is Worth Diffusion Guidance
Donghoon Ahn, Jiwon Kang, Sanghyun Lee +9
Diffusion models excel in generating high-quality images. However, current diffusion models struggle to produce reliable images without guidance methods, such as classifier-free gu…