54 citations · 87 across the 20 of their papers we have counts for
6 papers · 1 filter
Purify Before You Align: Improving Representation Alignment for Diffusion Models
Yingheng Wang, Yaoqiang Li, Yaqin Wu +4
Representation alignment improves and accelerates diffusion training by encouraging intermediate features to match a pretrained visual encoder. However, standard alignment matches…
Revisiting the 16x Pretraining Compute Gap between Diffusion and Autoregressive Language Models
Guanghan Wang, Bing He, Zhan Shi +3
Diffusion language models (DLMs) have attracted wide attention for their fast, parallel decoding, but they are believed to be far less efficient to train than autoregressive langua…
Clock Diffusion: Efficient Semi-Autoregressive Continuous Diffusion Language Models
Yair Schiff, Omer Belhasin, Roy Uziel +7
Recent works on continuous diffusion for discrete data have demonstrated performance on par with comparable discrete diffusion models. However, these continuous counterparts lack k…
Set Diffusion: Interpolating Token Orderings Between Autoregression and Diffusion for Fast and Flexible Decoding
Marianne Arriola, Volodymyr Kuleshov
Discrete diffusion models have steadily improved in quality relative to autoregressive (AR) models. However, these models are normally constrained to fixed-length generation and do…
DUEL: Exact Likelihood for Masked Diffusion via Deterministic Unmasking
Gilad Turok, Chris De Sa, Volodymyr Kuleshov
Masked diffusion models (MDMs) generate text by iteratively selecting positions to unmask and then predicting tokens at those positions. Yet MDMs lack proper likelihood evaluation:…
Learn from Your Mistakes: Self-Correcting Masked Diffusion Models
Yair Schiff, Omer Belhasin, Roy Uziel +6
Masked diffusion models (MDMs) have emerged as a promising alternative to autoregressive models, enabling parallel token generation while achieving competitive performance. Despite…