7 papers
Revise, Don't Freeze: Sampler-Matched Training for Self-Correcting Masked Diffusion Language Models
Longxuan Yu, Shaorong Zhang, Yu Fu +3
Masked diffusion language models (MDLMs) re-predict every position at each denoising step, but standard samplers commit tokens once revealed, leaving this revision capability unuse…
DSL-LLaDA: Scaling Continuous Denoising to 8B Masked Diffusion LMs
Longxuan Yu, Yunshu Wu, Yu Fu +5
Discrete Masked diffusion language models generate text by iterative parallel decoding, but few-step decoding suffers from a tradeoff between length and quality: with a fixed step…
Is Your Diffusion Sampler Actually Correct? A Sampler-Centric Evaluation of Discrete Diffusion Language Models
Luhan Tang, Longxuan Yu, Shaorong Zhang +1
Discrete diffusion language models (dLLMs) provide a fast and flexible alternative to autoregressive models (ARMs) via iterative denoising with parallel updates. However, their eva…
Generation Order and Parallel Decoding in Masked Diffusion Models: An Information-Theoretic Perspective
Shaorong Zhang, Longxuan Yu, Rob Brekelmans +3
Masked Diffusion Models (MDMs) significantly accelerate inference by trading off sequential determinism. However, the theoretical mechanisms governing generation order and the risk…
Thinking Out of Order: When Output Order Stops Reflecting Reasoning Order in Diffusion Language Models
Longxuan Yu, Yu Fu, Shaorong Zhang +4
Autoregressive (AR) language models enforce a fixed left-to-right generation order, creating a fundamental limitation when the required output structure conflicts with natural reas…
MMG: Mutual Information Estimation via the MMSE Gap in Diffusion
Longxuan Yu, Xing Shi, Xianghao Kong +2
Mutual information (MI) is one of the most general ways to measure relationships between random variables, but estimating this quantity for complex systems is challenging. Denoisin…