8 papers
Autoregressive Boltzmann Generators
Danyal Rehman, Charlie B. Tan, Yoshua Bengio +2
Efficient sampling of molecular systems at thermodynamic equilibrium is a hallmark challenge in statistical physics. This challenge has driven the development of Boltzmann Generato…
Hacking Generative Perplexity: Why Unconditional Text Evaluation Needs Distributional Metrics
Antonio Franca, Alexander Tong
Diffusion and continuous flow-based language models have emerged as the leading non-autoregressive alternatives to language modeling. Progress in both paradigms is overwhelmingly t…
Why Are DMD Students Lazy? Understanding the Copying Behavior in Few-Step Distillation
Shucheng Li, Iolo Jones, Alexander Tong +1
Distribution Matching Distillation (DMD) compresses pretrained diffusion models into efficient few-step generators by aligning their noised distributions across all scales. In prin…
Strong Stochastic Flow Maps
Sam McCallum, Zander W. Blasingame, Timothy Herschell +3
Flow and diffusion models generate high-quality samples in many modalities; however, many network evaluations are required during inference due to numerical integration of an under…
Coupling Models for One-Step Discrete Generation
Fred Zhangzhi Peng, Avishek Joey Bose, Anru R. Zhang +1
Generative modeling over discrete structures underpins applications across deep learning, from biological sequence design and code generation to large language models, yet generati…
Don't Retrain, Align: Adapting Autoregressive LMs to Diffusion LMs via Representation Alignment
Fred Zhangzhi Peng, Alexis Fox, Anru R. Zhang +1
Diffusion language models (DLMs) have recently demonstrated capabilities that complement standard autoregressive (AR) models, particularly in non-sequential generation and bidirect…