A Comparative analysis of Layer-wise Representational Capacity in AR and Diffusion LLMs

Raghavv Goel, Risheek Garrepalli, Sudhanshu Agrawal +3

Autoregressive (AR) language models build representations incrementally via left-to-right prediction, while diffusion language models (dLLMs) are trained through full-sequence deno…