works on
bidirectional generation 1hybrid attention 1language model fine-tuning 1masked diffusion language modeling 1pretrained autoregressive models 1

From the 1 of 1 linked paper with an AI index.

collaborators

1 paper

cs.AI2026

PreDiff-LM: Pretrained Discrete Masked Diffusion Language Modeling with Hybrid Attention

Zhengtao Yao, Runhao Li, Xupeng Chen +12

The paper presents PreDiff-LM, a discrete masked diffusion language model that retains causal attention on the prompt while applying bidirectional attention within masked targets,…