6 citations · 37 across the 30 of their papers we have counts for
15 papers · 1 filter
Rethinking the Trust Region in LLM Reinforcement Learning
Penghui Qi, Xiangxin Zhou, Zichen Liu +4
Reinforcement learning (RL) has become a cornerstone for fine-tuning Large Language Models (LLMs), with Proximal Policy Optimization (PPO) serving as the de facto standard algorith…
Defeating the Training-Inference Mismatch via FP16
Penghui Qi, Zichen Liu, Xiangxin Zhou +4
Reinforcement learning (RL) fine-tuning of large language models (LLMs) often suffers from instability due to the numerical mismatch between the training and inference policies. Wh…
Nonparametric Data Attribution for Diffusion Models
Yutian Zhao, Chao Du, Xiaosen Zheng +2
Data attribution for generative models seeks to quantify the influence of individual training examples on model outputs. Existing methods for diffusion models typically require acc…
Continual Reinforcement Learning by Planning with Online World Models
Zichen Liu, Guoji Fu, Chao Du +2
Continual reinforcement learning (CRL) refers to a naturalistic setting where an agent needs to endlessly evolve, by trial and error, to solve multiple tasks that are presented seq…
Reinforcing General Reasoning without Verifiers
Xiangxin Zhou, Zichen Liu, Anya Sims +6
The recent paradigm shift towards training large language models (LLMs) using DeepSeek-R1-Zero-style reinforcement learning (RL) on verifiable rewards has led to impressive advance…
Optimizing Anytime Reasoning via Budget Relative Policy Optimization
Penghui Qi, Zichen Liu, Tianyu Pang +3
Scaling test-time compute is crucial for enhancing the reasoning capabilities of large language models (LLMs). Existing approaches typically employ reinforcement learning (RL) to m…
Understanding R1-Zero-Like Training: A Critical Perspective
Zichen Liu, Changyu Chen, Wenjun Li +5
DeepSeek-R1-Zero has shown that reinforcement learning (RL) at scale can directly enhance the reasoning capabilities of LLMs without supervised fine-tuning. In this work, we critic…
Sample-Efficient Alignment for LLMs
Zichen Liu, Changyu Chen, Chao Du +2
We study methods for efficiently aligning large language models (LLMs) with human preferences given budgeted online feedback. We first formulate the LLM alignment problem in the fr…
Improved Techniques for Optimization-Based Jailbreaking on Large Language Models
Xiaojun Jia, Tianyu Pang, Chao Du +5
Large language models (LLMs) are being rapidly developed, and a key component of their widespread deployment is their safety-related alignment. Many red-teaming efforts aim to jail…
Graph Diffusion Policy Optimization
Yijing Liu, Chao Du, Tianyu Pang +3
Recent research has made significant progress in optimizing diffusion models for downstream objectives, which is an important pursuit in fields such as graph generation for drug de…
Benchmarking Large Multimodal Models against Common Corruptions
Jiawei Zhang, Tianyu Pang, Chao Du +3
This technical report aims to fill a deficiency in the assessment of large multimodal models (LMMs) by specifically examining the self-consistency of their outputs when subjected t…
Locality Sensitive Sparse Encoding for Learning World Models Online
Zichen Liu, Chao Du, Wee Sun Lee +1
Acquiring an accurate world model online for model-based reinforcement learning (MBRL) is challenging due to data nonstationarity, which typically causes catastrophic forgetting fo…
Finetuning Text-to-Image Diffusion Models for Fairness
Xudong Shen, Chao Du, Tianyu Pang +3
The rapid adoption of text-to-image diffusion models in society underscores an urgent need to address their biases. Without interventions, these biases could propagate a skewed wor…
Intriguing Properties of Data Attribution on Diffusion Models
Xiaosen Zheng, Tianyu Pang, Chao Du +2
Data attribution seeks to trace model outputs back to training data. With the recent development of diffusion models, data attribution has become a desired module to properly assig…
On Memorization in Diffusion Models
Xiangming Gu, Chao Du, Tianyu Pang +3
Due to their capacity to generate novel and high-quality samples, diffusion models have attracted significant research interest in recent years. Notably, the typical training objec…