Showing cs.CLShow all
2 papers · 1 filter
cs.CL2025
SPG: Sandwiched Policy Gradient for Masked Diffusion Language Models
Chenyu Wang, Paria Rashidinejad, DiJia Su +9
Diffusion large language models (dLLMs) are emerging as an efficient alternative to autoregressive models due to their ability to decode multiple tokens in parallel. However, align…
cs.CL2025
Positional Encoding via Token-Aware Phase Attention
Yu Wang, Sheng Shen, Rémi Munos +2
We prove under practical assumptions that Rotary Positional Embedding (RoPE) introduces an intrinsic distance-dependent bias in attention scores that limits RoPE's ability to model…