8 papers
OPD+: Rethinking the Advantage Design for On-Policy Distillation
Hanyang Zhao, Haoxian Chen, Han Lin +3
On-policy distillation (OPD) is a widely used technique to transfer capabilities from capable teacher language models to the base student models, and can be formulated in a reinfor…
RPO: Fine-Tuning Visual Generative Models via Rich Vision-Language Preferences
Hanyang Zhao, Haoxian Chen, Yucheng Guo +5
Traditional preference tuning methods for LLMs/Visual Generative Models often rely solely on reward model labeling, which can be opaque, offer limited insights into the rationale b…
Understanding Sampler Stochasticity in Training Diffusion Models for RLHF
Jiayuan Sheng, Hanyang Zhao, Haoxian Chen +2
Reinforcement Learning from Human Feedback (RLHF) is increasingly used to fine-tune diffusion models, but a key challenge arises from the mismatch between stochastic samplers used…
SOCRATES: Simulation Optimization with Correlated Replicas and Adaptive Trajectory Evaluations
Haoting Zhang, Haoxian Chen, Donglin Zhan +5
The field of simulation optimization (SO) encompasses various methods developed to optimize complex, expensive-to-sample stochastic systems. Established methods include, but are no…
Score as Action: Fine-Tuning Diffusion Generative Models by Continuous-time Reinforcement Learning
Hanyang Zhao, Haoxian Chen, Ji Zhang +2
Reinforcement learning from human feedback (RLHF), which aligns a diffusion model with input prompt, has become a crucial step in building reliable generative AI models. Most works…
Prediction-Enhanced Monte Carlo: A Machine Learning View on Control Variate
Fengpei Li, Haoxian Chen, Jiahe Lin +7
For many complex simulation tasks spanning areas such as healthcare, engineering, and finance, Monte Carlo (MC) methods are invaluable due to their unbiased estimates and precise e…