5 papers
Clock Diffusion: Efficient Semi-Autoregressive Continuous Diffusion Language Models
Yair Schiff, Omer Belhasin, Roy Uziel +7
Recent works on continuous diffusion for discrete data have demonstrated performance on par with comparable discrete diffusion models. However, these continuous counterparts lack k…
DUEL: Exact Likelihood for Masked Diffusion via Deterministic Unmasking
Gilad Turok, Chris De Sa, Volodymyr Kuleshov
Masked diffusion models (MDMs) generate text by iteratively selecting positions to unmask and then predicting tokens at those positions. Yet MDMs lack proper likelihood evaluation:…
Learn from Your Mistakes: Self-Correcting Masked Diffusion Models
Yair Schiff, Omer Belhasin, Roy Uziel +6
Masked diffusion models (MDMs) have emerged as a promising alternative to autoregressive models, enabling parallel token generation while achieving competitive performance. Despite…
d2: Improving Reasoning in Diffusion Language Models via Trajectory Likelihood Estimation
Guanghan Wang, Gilad Turok, Yair Schiff +2
While diffusion language models (DLMs) have achieved competitive performance in text generation, improving their reasoning ability with reinforcement learning remains an active res…
Sampling From Multiscale Densities With Delayed Rejection Generalized Hamiltonian Monte Carlo
Gilad Turok, Chirag Modi, Bob Carpenter
Hamiltonian Monte Carlo (HMC) is the mainstay of applied Bayesian inference for differentiable models. However, HMC still struggles to sample from hierarchical models that induce d…