On Trajectory-Aware Training for Masked Diffusion Language Models

Manuel Madeira, Amitis Shidani, Alice Bizeul +6

Masked diffusion models (MDMs) generate text by unmasking several tokens per step, but they are trained and sampled under different conditions. The model is trained on randomly mas…