Decoding in Order-Agnostic Language Models: Chain-Rule Deviation and Uniform Spreading

Lin Yao

Order-agnostic language models (OALMs), including discrete diffusion language models (dLLMs), are trained to predict masked tokens under arbitrary conditioning sets, allowing seque…