127 citations · 335 across the 11 of their papers we have counts for
11 papers
Aligner: Efficient Alignment by Learning to Correct
Jiaming Ji, Boyuan Chen, Hantao Lou +6
With the rapid development of large language models (LLMs) and ever-evolving practical requirements, finding an efficient and effective alignment method has never been more critica…
Safe RLHF: Safe Reinforcement Learning from Human Feedback
Josef Dai, Xuehai Pan, Ruiyang Sun +5
With the development of large language models (LLMs), striking a balance between the performance and safety of AI systems has never been more critical. However, the inherent tensio…
AI Alignment: A Comprehensive Survey
Jiaming Ji, Tianyi Qiu, Boyuan Chen +23
AI alignment aims to make AI systems behave in line with human intentions and values. As AI systems grow more capable, so do risks from misalignment. To provide a comprehensive and…
Safety-Gymnasium: A Unified Safe Reinforcement Learning Benchmark
Jiaming Ji, Borong Zhang, Jiayi Zhou +7
Artificial intelligence (AI) systems possess significant potential to drive societal progress. However, their deployment often faces obstacles due to substantial safety concerns. S…
Baichuan 2: Open Large-scale Language Models
Aiyuan Yang, Bin Xiao, Bingning Wang +52
Large language models (LLMs) have demonstrated remarkable performance on a variety of natural language tasks based on just a few examples of natural language instructions, reducing…
SafeDreamer: Safe Reinforcement Learning with World Models
Weidong Huang, Jiaming Ji, Chunhe Xia +2
The deployment of Reinforcement Learning (RL) in real-world applications is constrained by its failure to satisfy safety criteria. Existing Safe Reinforcement Learning (SafeRL) met…
BeaverTails: Towards Improved Safety Alignment of LLM via a Human-Preference Dataset
Jiaming Ji, Mickel Liu, Juntao Dai +6
In this paper, we introduce the BeaverTails dataset, aimed at fostering research on safety alignment in large language models (LLMs). This dataset uniquely separates annotations of…
OmniSafe: An Infrastructure for Accelerating Safe Reinforcement Learning Research
Jiaming Ji, Jiayi Zhou, Borong Zhang +7
AI systems empowered by reinforcement learning (RL) algorithms harbor the immense potential to catalyze societal advancement, yet their deployment is often impeded by significant s…
Heterogeneous-Agent Reinforcement Learning
Yifan Zhong, Jakub Grudzien Kuba, Xidong Feng +3
The necessity for cooperation among intelligent machines has popularised cooperative multi-agent reinforcement learning (MARL) in AI research. However, many research endeavours hea…
Constrained Update Projection Approach to Safe Policy Optimization
Long Yang, Jiaming Ji, Juntao Dai +5
Safe reinforcement learning (RL) studies problems where an intelligent agent has to not only maximize reward but also avoid exploring unsafe areas. In this study, we propose CUP, a…
CUP: A Conservative Update Policy Algorithm for Safe Reinforcement Learning
Long Yang, Jiaming Ji, Juntao Dai +3
Safe reinforcement learning (RL) is still very challenging since it requires the agent to consider both return maximization and safe exploration. In this paper, we propose CUP, a C…