From the 1 of 12 linked papers with an AI index.
12 papers
A Continuous-Time Reinforcement Learning Framework for Fine-Tuning Discrete Diffusion Models
Zikun Zhang, Jiayuan Sheng, David D. Yao +1
The paper introduces a continuous‑time reinforcement‑learning framework for fine‑tuning discrete diffusion models, especially masked diffusion language models, by modeling state dy…
OPD+: Rethinking the Advantage Design for On-Policy Distillation
Hanyang Zhao, Haoxian Chen, Han Lin +3
On-policy distillation (OPD) is a widely used technique to transfer capabilities from capable teacher language models to the base student models, and can be formulated in a reinfor…
Improved techniques for fine-tuning flow models via adjoint matching: a deterministic control pipeline
Zhengyi Guo, Jiayuan Sheng, David D. Yao +1
We propose a deterministic adjoint matching framework that formulates human preference alignment for flow-based generative models as an optimal control problem over velocity fields…
DiFFPO: Training Diffusion LLMs to Reason Fast and Furious via Reinforcement Learning
Hanyang Zhao, Dawen Liang, Wenpin Tang +2
We propose DiFFPO, Diffusion Fast and Furious Policy Optimization, a unified framework for training masked diffusion large language models (dLLMs) to reason not only better (furiou…
RPO: Fine-Tuning Visual Generative Models via Rich Vision-Language Preferences
Hanyang Zhao, Haoxian Chen, Yucheng Guo +5
Traditional preference tuning methods for LLMs/Visual Generative Models often rely solely on reward model labeling, which can be opaque, offer limited insights into the rationale b…
Understanding Sampler Stochasticity in Training Diffusion Models for RLHF
Jiayuan Sheng, Hanyang Zhao, Haoxian Chen +2
Reinforcement Learning from Human Feedback (RLHF) is increasingly used to fine-tune diffusion models, but a key challenge arises from the mismatch between stochastic samplers used…