Less Uniform Discrete Diffusion is More Powerful and Scalable

Kaibo Wang, Ding Ding, Fangyu Ding +5

Although uniform diffusion language models (UDLMs) represent a promising diffusion paradigm, scaling them remains challenging. We identify the core obstacle as an over-uniform trai…