#hybrid attention

topichybrid attention

1 paper · 1 filter

cs.AI2026

PreDiff-LM: Pretrained Discrete Masked Diffusion Language Modeling with Hybrid Attention

Zhengtao Yao, Runhao Li, Xupeng Chen +12

The paper presents PreDiff-LM, a discrete masked diffusion language model that retains causal attention on the prompt while applying bidirectional attention within masked targets,…