Showing cs.CLShow all
2 papers · 1 filter
cs.CL2025
Large Language Diffusion Models
Shen Nie, Fengqi Zhu, Zebin You +7
The capabilities of large language models (LLMs) are widely regarded as relying on autoregressive models (ARMs). We challenge this notion by introducing LLaDA, a diffusion model tr…
cs.CL2025
LLaDA-MoE: A Sparse MoE Diffusion Language Model
Fengqi Zhu, Zebin You, Yipeng Xing +23
We introduce LLaDA-MoE, a large language diffusion model with the Mixture-of-Experts (MoE) architecture, trained from scratch on approximately 20T tokens. LLaDA-MoE achieves compet…