127 citations · 309 across the 14 of their papers we have counts for
8 papers · 1 filter
Safe RLHF: Safe Reinforcement Learning from Human Feedback
Josef Dai, Xuehai Pan, Ruiyang Sun +5
With the development of large language models (LLMs), striking a balance between the performance and safety of AI systems has never been more critical. However, the inherent tensio…
AI Alignment: A Comprehensive Survey
Jiaming Ji, Tianyi Qiu, Boyuan Chen +23
AI alignment aims to make AI systems behave in line with human intentions and values. As AI systems grow more capable, so do risks from misalignment. To provide a comprehensive and…
Safety-Gymnasium: A Unified Safe Reinforcement Learning Benchmark
Jiaming Ji, Borong Zhang, Jiayi Zhou +7
Artificial intelligence (AI) systems possess significant potential to drive societal progress. However, their deployment often faces obstacles due to substantial safety concerns. S…
Evolving Diverse Red-team Language Models in Multi-round Multi-agent Games
Chengdong Ma, Ziran Yang, Hai Ci +4
The primary challenge in deploying Large Language Model (LLM) is ensuring its harmlessness. Red team can identify vulnerabilities by attacking LLM to attain safety. However, curren…
Baichuan 2: Open Large-scale Language Models
Aiyuan Yang, Bin Xiao, Bingning Wang +52
Large language models (LLMs) have demonstrated remarkable performance on a variety of natural language tasks based on just a few examples of natural language instructions, reducing…
BeaverTails: Towards Improved Safety Alignment of LLM via a Human-Preference Dataset
Jiaming Ji, Mickel Liu, Juntao Dai +6
In this paper, we introduce the BeaverTails dataset, aimed at fostering research on safety alignment in large language models (LLMs). This dataset uniquely separates annotations of…
OmniSafe: An Infrastructure for Accelerating Safe Reinforcement Learning Research
Jiaming Ji, Jiayi Zhou, Borong Zhang +7
AI systems empowered by reinforcement learning (RL) algorithms harbor the immense potential to catalyze societal advancement, yet their deployment is often impeded by significant s…
Proactive Multi-Camera Collaboration For 3D Human Pose Estimation
Hai Ci, Mickel Liu, Xuehai Pan +2
This paper presents a multi-agent reinforcement learning (MARL) scheme for proactive Multi-Camera Collaboration in 3D Human Pose Estimation in dynamic human crowds. Traditional fix…