MDLMPE: Distribution Aware Positional Encoding for Masked Diffusion Language Models

Tong Ling, Hang Lei, Feng Xiao +5

Masked diffusion language models (MDLMs) enable parallel generation and bidirectional context modeling, but their positional context differs fundamentally from that of autoregressi…