2 citations · 3 across the 14 of their papers we have counts for
15 papers
JustOPD: On-Policy Distillation from Autoregressive to Diffusion Language Models
Zikun Zhang, Jiayuan Sheng, David D. Yao +1
On-policy distillation (OPD) is a post-training approach for large language models (LLMs) that enables a student model to learn a teacher model's distribution from sequences sample…
Spec2Twin-Chain: Orchestrating Bi-Level Optimization with LLMs for Blockchain Digital Twin Construction
Haoting Zhang, Haoxian Chen, Jiayuan Sheng +4
Building a blockchain digital twin largely requires translating domain knowledge and specific system descriptions into a simulator architecture, calibrating its parameters against…
A Continuous-Time Reinforcement Learning Framework for Fine-Tuning Discrete Diffusion Models
Zikun Zhang, Jiayuan Sheng, David D. Yao +1
We formulate reinforcement learning (RL) in continuous time with discrete state spaces and possibly arbitrary action spaces via a stochastic control approach, where the state dynam…
OPD+: Rethinking the Advantage Design for On-Policy Distillation
Hanyang Zhao, Haoxian Chen, Han Lin +3
On-policy distillation (OPD) is a widely used technique to transfer capabilities from capable teacher language models to the base student models, and can be formulated in a reinfor…
Improved techniques for fine-tuning flow models via adjoint matching: a deterministic control pipeline
Zhengyi Guo, Jiayuan Sheng, David D. Yao +1
We propose a deterministic adjoint matching framework that formulates human preference alignment for flow-based generative models as an optimal control problem over velocity fields…
SOCRATES: Simulation Optimization with Correlated Replicas and Adaptive Trajectory Evaluations
Haoting Zhang, Haoxian Chen, Donglin Zhan +5
The field of simulation optimization (SO) encompasses various methods developed to optimize complex, expensive-to-sample stochastic systems. Established methods include, but are no…
Understanding Sampler Stochasticity in Training Diffusion Models for RLHF
Jiayuan Sheng, Hanyang Zhao, Haoxian Chen +2
Reinforcement Learning from Human Feedback (RLHF) is increasingly used to fine-tune diffusion models, but a key challenge arises from the mismatch between stochastic samplers used…
DiFFPO: Training Diffusion LLMs to Reason Fast and Furious via Reinforcement Learning
Hanyang Zhao, Dawen Liang, Wenpin Tang +2
We propose DiFFPO, Diffusion Fast and Furious Policy Optimization, a unified framework for training masked diffusion large language models (dLLMs) to reason not only better (furiou…
Diffusion Generative Models Meet Compressed Sensing, with Applications to Imaging and Finance
Zhengyi Guo, Jiatu Li, Wenpin Tang +1
In this study we develop dimension-reduction techniques to accelerate diffusion model inference in the context of synthetic data generation. The idea is to integrate compressed sen…
RPO: Fine-Tuning Visual Generative Models via Rich Vision-Language Preferences
Hanyang Zhao, Haoxian Chen, Yucheng Guo +5
Traditional preference tuning methods for LLMs/Visual Generative Models often rely solely on reward model labeling, which can be opaque, offer limited insights into the rationale b…
Score as Action: Fine-Tuning Diffusion Generative Models by Continuous-time Reinforcement Learning
Hanyang Zhao, Haoxian Chen, Ji Zhang +2
Reinforcement learning from human feedback (RLHF), which aligns a diffusion model with input prompt, has become a crucial step in building reliable generative AI models. Most works…
RainbowPO: A Unified Framework for Combining Improvements in Preference Optimization
Hanyang Zhao, Genta Indra Winata, Anirban Das +4
Recently, numerous preference optimization algorithms have been introduced as extensions to the Direct Preference Optimization (DPO) family. While these methods have successfully a…
Scores as Actions: a framework of fine-tuning diffusion models by continuous-time reinforcement learning
Hanyang Zhao, Haoxian Chen, Ji Zhang +2
Reinforcement Learning from human feedback (RLHF) has been shown a promising direction for aligning generative models with human intent and has also been explored in recent works f…
Preference Tuning with Human Feedback on Language, Speech, and Vision Tasks: A Survey
Genta Indra Winata, Hanyang Zhao, Anirban Das +4
Preference tuning is a crucial process for aligning deep generative models with human preferences. This survey offers a thorough overview of recent advancements in preference tunin…
MallowsPO: Fine-Tune Your LLM with Preference Dispersions
Haoxian Chen, Hanyang Zhao, Henry Lam +2
Direct Preference Optimization (DPO) has recently emerged as a popular approach to improve reinforcement learning with human feedback (RLHF), leading to better techniques to fine-t…