2 citations · 5 across the 16 of their papers we have counts for
12 papers · 1 filter
JustOPD: On-Policy Distillation from Autoregressive to Diffusion Language Models
Zikun Zhang, Jiayuan Sheng, David D. Yao +1
On-policy distillation (OPD) is a post-training approach for large language models (LLMs) that enables a student model to learn a teacher model's distribution from sequences sample…
DOHF: Online Diffusion Fine-tuning with Doob's -transform Guidance
Zhengyi Guo, Jiayuan Sheng, Wenpin Tang +1
Reward-based diffusion fine-tuning faces practical challenges when desirable outcomes are rare or conditioning corrections are costly to estimate. In this work, we propose Diffusio…
A Continuous-Time Reinforcement Learning Framework for Fine-Tuning Discrete Diffusion Models
Zikun Zhang, Jiayuan Sheng, David D. Yao +1
We formulate reinforcement learning (RL) in continuous time with discrete state spaces and possibly arbitrary action spaces via a stochastic control approach, where the state dynam…
OPD+: Rethinking the Advantage Design for On-Policy Distillation
Hanyang Zhao, Haoxian Chen, Han Lin +3
On-policy distillation (OPD) is a widely used technique to transfer capabilities from capable teacher language models to the base student models, and can be formulated in a reinfor…
Understanding Sampler Stochasticity in Training Diffusion Models for RLHF
Jiayuan Sheng, Hanyang Zhao, Haoxian Chen +2
Reinforcement Learning from Human Feedback (RLHF) is increasingly used to fine-tune diffusion models, but a key challenge arises from the mismatch between stochastic samplers used…
DiFFPO: Training Diffusion LLMs to Reason Fast and Furious via Reinforcement Learning
Hanyang Zhao, Dawen Liang, Wenpin Tang +2
We propose DiFFPO, Diffusion Fast and Furious Policy Optimization, a unified framework for training masked diffusion large language models (dLLMs) to reason not only better (furiou…
RPO: Fine-Tuning Visual Generative Models via Rich Vision-Language Preferences
Hanyang Zhao, Haoxian Chen, Yucheng Guo +5
Traditional preference tuning methods for LLMs/Visual Generative Models often rely solely on reward model labeling, which can be opaque, offer limited insights into the rationale b…
Score as Action: Fine-Tuning Diffusion Generative Models by Continuous-time Reinforcement Learning
Hanyang Zhao, Haoxian Chen, Ji Zhang +2
Reinforcement learning from human feedback (RLHF), which aligns a diffusion model with input prompt, has become a crucial step in building reliable generative AI models. Most works…
Scores as Actions: a framework of fine-tuning diffusion models by continuous-time reinforcement learning
Hanyang Zhao, Haoxian Chen, Ji Zhang +2
Reinforcement Learning from human feedback (RLHF) has been shown a promising direction for aligning generative models with human intent and has also been explored in recent works f…
MallowsPO: Fine-Tune Your LLM with Preference Dispersions
Haoxian Chen, Hanyang Zhao, Henry Lam +2
Direct Preference Optimization (DPO) has recently emerged as a popular approach to improve reinforcement learning with human feedback (RLHF), leading to better techniques to fine-t…
Score-based Diffusion Models via Stochastic Differential Equations -- a Technical Tutorial
Wenpin Tang, Hanyang Zhao
This is an expository article on the score-based diffusion models, with a particular focus on the formulation via stochastic differential equations (SDE). After a gentle introducti…
Contractive Diffusion Probabilistic Models
Wenpin Tang, Hanyang Zhao
Diffusion probabilistic models (DPMs) have emerged as a promising technique in generative modeling. The success of DPMs relies on two ingredients: time reversal of diffusion proces…