5 papers
Does Uniform Discrete Diffusion Need Time?
Chunsan Hong, Chieh-Hsin Lai, Satoshi Hayakawa +3
Uniform discrete diffusion models (UDMs) commonly use explicit time conditioning, but we find that it can often be unnecessary in practice. In this paper, we first show that the po…
Looped Diffusion Language Models
Sanghyun Lee, Chunsan Hong, Seungryong Kim +3
Masked diffusion models (MDMs) have emerged as a promising alternative to autoregressive models for language modeling, yet the effective design of transformer architectures for MDM…
Understanding and Accelerating the Training of Masked Diffusion Language Models
Chunsan Hong, Sanghyun Lee, Chieh-Hsin Lai +5
Masked diffusion models (MDMs) have emerged as a promising alternative to autoregressive models (ARMs) for language modeling. However, MDMs are known to learn substantially more sl…
Unifying Masked Diffusion Models with Various Generation Orders and Beyond
Chunsan Hong, Sanghyun Lee, Jong Chul Ye
Masked diffusion models (MDMs) are a potential alternative to autoregressive models (ARMs) for language generation, but generation quality depends critically on the generation orde…
Improving Discrete Diffusion Unmasking Policies Beyond Explicit Reference Policies
Chunsan Hong, Seonho An, Min-Soo Kim +1
Masked diffusion models (MDMs) have recently emerged as a novel framework for language modeling. MDMs generate sentences by iteratively denoising masked sequences, filling in [MASK…