29 citations · 43 across the 13 of their papers we have counts for
13 papers
One Model for All: Multi-Objective Controllable Language Models
Qiang He, Yucheng Yang, Tianyi Zhou +3
Aligning large language models (LLMs) with human preferences is critical for enhancing LLMs' safety, helpfulness, humor, faithfulness, etc. Current reinforcement learning from huma…
Scaling Behaviors of LLM Reinforcement Learning Post-Training: An Empirical Study in Mathematical Reasoning
Zelin Tan, Hejia Geng, Xiaohang Yu +14
While scaling laws for large language models (LLMs) during pre-training have been extensively studied, their behavior under reinforcement learning (RL) post-training remains largel…
Pareto Multi-Objective Alignment for Language Models
Qiang He, Setareh Maghsudi
Large language models (LLMs) are increasingly deployed in real-world applications that require careful balancing of multiple, often conflicting, objectives, such as informativeness…
Task Adaptation from Skills: Information Geometry, Disentanglement, and New Objectives for Unsupervised Reinforcement Learning
Yucheng Yang, Tianyi Zhou, Qiang He +3
Unsupervised reinforcement learning (URL) aims to learn general skills for unseen downstream tasks. Mutual Information Skill Learning (MISL) addresses URL by maximizing the mutual…
Advancing DRL Agents in Commercial Fighting Games: Training, Integration, and Agent-Human Alignment
Chen Zhang, Qiang He, Zhou Yuan +4
Deep Reinforcement Learning (DRL) agents have demonstrated impressive success in a wide range of game genres. However, existing research primarily focuses on optimizing DRL compete…
Adaptive Regularization of Representation Rank as an Implicit Constraint of Bellman Equation
Qiang He, Tianyi Zhou, Meng Fang +1
Representation rank is an important concept for understanding the role of Neural Networks (NNs) in Deep Reinforcement learning (DRL), which measures the expressive capacity of valu…
Keep Various Trajectories: Promoting Exploration of Ensemble Policies in Continuous Control
Chao Li, Chen Gong, Qiang He +1
The combination of deep reinforcement learning (DRL) with ensemble methods has been proved to be highly effective in addressing complex sequential decision-making problems. This su…
Eigensubspace of Temporal-Difference Dynamics and How It Improves Value Approximation in Reinforcement Learning
Qiang He, Tianyi Zhou, Meng Fang +1
We propose a novel value approximation method, namely Eigensubspace Regularized Critic (ERC) for deep reinforcement learning (RL). ERC is motivated by an analysis of the dynamics o…
Frustratingly Easy Regularization on Representation Can Boost Deep Reinforcement Learning
Qiang He, Huangyuan Su, Jieyu Zhang +1
Deep reinforcement learning (DRL) gives the promise that an agent learns good policy from high-dimensional information, whereas representation learning removes irrelevant and redun…
The -Divergence Reinforcement Learning Framework
Chen Gong, Qiang He, Yunpeng Bai +6
The framework of deep reinforcement learning (DRL) provides a powerful and widely applicable mathematical formalization for sequential decision-making. This paper present a novel D…
MEPG: A Minimalist Ensemble Policy Gradient Framework for Deep Reinforcement Learning
Qiang He, Huangyuan Su, Chen Gong +1
During the training of a reinforcement learning (RL) agent, the distribution of training data is non-stationary as the agent's behavior changes over time. Therefore, there is a ris…
POPO: Pessimistic Offline Policy Optimization
Qiang He, Xinwen Hou
Offline reinforcement learning (RL), also known as batch RL, aims to optimize policy from a large pre-recorded dataset without interaction with the environment. This setting offers…
WD3: Taming the Estimation Bias in Deep Reinforcement Learning
Qiang He, Xinwen Hou
The overestimation phenomenon caused by function approximation is a well-known issue in value-based reinforcement learning algorithms such as deep Q-networks and DDPG, which could…