13 papers
Scaling Reinforcement Learning for Diffusion Models via Velocity Matching
Jaemoo Choi, Wei Guo, Yuchen Zhu +4
Reward-based fine-tuning of diffusion models has largely inherited likelihood-based policy optimization developed for autoregressive large language models (LLMs). Diffusion models,…
MetaDNS: Enhancing Exploration in Discrete Neural Samplers via Well-Tempered Metadynamics
Xiaochen Du, Juno Nam, Jaemoo Choi +7
Sampling from discrete distributions with multiple modes and energy barriers is fundamental to machine learning and computational physics. Recent discrete neural samplers like MDNS…
Efficient Adjoint Matching for Fine-tuning Diffusion Models
Jeongwoo Shin, Dongsoo Shin, Yuchen Zhu +5
Reward fine-tuning has become a common approach for aligning pretrained diffusion and flow models with human preferences in text-to-image generation. Among reward-gradient-based me…
Continuous Diffusion Scales Competitively with Discrete Diffusion for Language
Zhihan Yang, Wei Guo, Shuibai Zhang +5
While diffusion has drawn considerable recent attention from the language modeling community, continuous diffusion has appeared less scalable than discrete approaches. To challenge…
Discrete Adjoint Schrödinger Bridge Sampler
Wei Guo, Yuchen Zhu, Xiaochen Du +6
Learning discrete neural samplers is challenging due to the lack of gradients and combinatorial complexity. While stochastic optimal control (SOC) and Schrödinger bridge (SB) provi…
Rethinking the Design Space of Reinforcement Learning for Diffusion Models: On the Importance of Likelihood Estimation Beyond Loss Design
Jaemoo Choi, Yuchen Zhu, Wei Guo +6
Reinforcement learning has been widely applied to diffusion and flow models for visual tasks such as text-to-image generation. However, these tasks remain challenging because diffu…
Proximal Diffusion Neural Sampler
Wei Guo, Jaemoo Choi, Yuchen Zhu +2
The task of learning a diffusion-based neural sampler for drawing samples from an unnormalized target distribution can be viewed as a stochastic optimal control problem on path mea…
MDNS: Masked Diffusion Neural Sampler via Stochastic Optimal Control
Yuchen Zhu, Wei Guo, Jaemoo Choi +3
We study the problem of learning a neural sampler to generate samples from discrete state spaces where the target probability mass function is known up to…
Ctrl-Z Sampling: Scaling Diffusion Sampling with Controlled Random Zigzag Explorations
Shunqi Mao, Wei Guo, Chaoyi Zhang +3
Diffusion models generate conditional samples by progressively denoising Gaussian noise, yet the denoising trajectory can stall at visually plausible but low-quality outcomes with…
Complexity Analysis of Normalizing Constant Estimation: from Jarzynski Equality to Annealed Importance Sampling and beyond
Wei Guo, Molei Tao, Yongxin Chen
Given an unnormalized probability density , estimating its normalizing constant or free energy $F=-\lo…
Fast Solvers for Discrete Diffusion Models: Theory and Applications of High-Order Algorithms
Yinuo Ren, Haoxuan Chen, Yuchen Zhu +5
Discrete diffusion models have emerged as a powerful generative modeling framework for discrete data with successful applications spanning from text generation to image synthesis.…
Plug-and-Play Controllable Generation for Discrete Masked Models
Wei Guo, Yuchen Zhu, Molei Tao +1
This article makes discrete masked models for the generative modeling of discrete data controllable. The goal is to generate samples of a discrete random variable that adheres to a…
Provable Benefit of Annealed Langevin Monte Carlo for Non-log-concave Sampling
Wei Guo, Molei Tao, Yongxin Chen
We consider the outstanding problem of sampling from an unnormalized density that may be non-log-concave and multimodal. To enhance the performance of simple Markov chain Monte Car…