7 papers
Unlearning in Diffusion Models: A Unified Framework with KL Divergence and Likelihood Constraints
Shervin Khalafi, Alejandro Ribeiro, Dongsheng Ding
Unlearning in diffusion models aims to remove undesirable data or concepts while preserving the utility of pretrained models -- two fundamentally conflicting objectives. We propose…
Where to Spend Rollouts: Hit-Utility Optimal Rollout Allocation for Group-Based RLVR
Tao Wang, Shuo Li, Yan Sun +2
Reinforcement learning with verifiable rewards (RLVR) has emerged as a central paradigm for improving the reasoning capabilities of large language models. Group-based policy optimi…
Composition and Alignment of Diffusion Models using Constrained Learning
Shervin Khalafi, Ignacio Hounie, Dongsheng Ding +1
Diffusion models have become prevalent in generative modeling due to their ability to sample from complex distributions. To improve the quality of generated samples and their compl…
Alignment of large language models with constrained learning
Botong Zhang, Shuo Li, Ignacio Hounie +3
We study the problem of computing an optimal large language model (LLM) policy for the constrained alignment problem, where the goal is to maximize a primary reward objective while…
Deterministic Policy Gradient Primal-Dual Methods for Continuous-Space Constrained MDPs
Sergio Rozada, Dongsheng Ding, Antonio G. Marques +1
We study the problem of computing deterministic optimal policies for constrained Markov decision processes (MDPs) with continuous state and action spaces, which are widely encounte…
Constrained Diffusion Models via Dual Training
Shervin Khalafi, Dongsheng Ding, Alejandro Ribeiro
Diffusion models have attained prominence for their ability to synthesize a probability distribution for a given dataset via a diffusion process, enabling the generation of new dat…