E-MoE: Enhanced Mixture-of-Experts for Non-Factorized Diffusion Language Models

Arseny Ivanov, Alexander Kolesov, Alexander Korotin +2

Masked diffusion models (MDMs) generate sequences by progressively unmasking several tokens per denoising step, but their reverse process is typically factorized over positions, li…