6 citations · 7 across the 10 of their papers we have counts for
7 papers · 1 filter
Risk-Conditioned Fine-Tuning of Large Language Models
Zixuan Liu, Fangzheng Wu, Brian Summa +2
Large Language Models (LLMs) are increasingly deployed in settings where rare but severe harmful generations can have significant consequences. Existing Risk-Averse RLHF addresses…
Adaptive Multilevel Twisted Sequential Monte Carlo for Rare Events Estimation in Language Models
Zixuan Liu, Fangzheng Wu, Brian Summa +1
Rare unsafe behaviors in large language models can remain practically significant even when their probability is extremely small, particularly at deployment scales involving millio…
Robust General Utility for Reinforcement Learning
Zixuan Liu, Fangzheng Wu, Brian Summa +1
Reinforcement learning (RL) with general utility extends classic RL by optimizing an arbitrary utility functional of the policy-induced occupancy measure, thereby enabling a broade…
Robust Optimization for Mitigating Reward Hacking with Correlated Proxies
Zixuan Liu, Xiaolin Sun, Zizhan Zheng
Designing robust reinforcement learning (RL) agents in the presence of imperfect reward signals remains a core challenge. In practice, agents are often trained with proxy rewards t…
What Makes Value Learning Efficient in Residual Reinforcement Learning?
Guozheng Ma, Lu Li, Haoyu Wang +3
Residual reinforcement learning (RL) enables stable online refinement of expressive pretrained policies by freezing the base and learning only bounded corrections. However, value l…
Enhancing LLM Safety via Constrained Direct Preference Optimization
Zixuan Liu, Xiaolin Sun, Zizhan Zheng
The rapidly increasing capabilities of large language models (LLMs) raise an urgent need to align AI systems with diverse human preferences to simultaneously enhance their usefulne…
Revisiting Plasticity in Visual Reinforcement Learning: Data, Modules and Training Stages
Guozheng Ma, Lu Li, Sen Zhang +6
Plasticity, the ability of a neural network to evolve with new data, is crucial for high-performance and sample-efficient visual reinforcement learning (VRL). Although methods like…