76 citations · 152 across the 39 of their papers we have counts for
9 papers · 1 filter
Align Anything: Training All-Modality Models to Follow Instructions with Language Feedback
Jiaming Ji, Jiayi Zhou, Hantao Lou +16
Reinforcement learning from human feedback (RLHF) has proven effective in enhancing the instruction-following capabilities of large language models; however, it remains underexplor…
Libra-Leaderboard: Towards Responsible AI through a Balanced Leaderboard of Safety and Capability
Haonan Li, Xudong Han, Zenan Zhai +32
To address this gap, we introduce Libra-Leaderboard, a comprehensive framework designed to rank LLMs through a balanced evaluation of performance and safety. Combining a dynamic le…
Sequence to Sequence Reward Modeling: Improving RLHF by Language Feedback
Jiayi Zhou, Jiaming Ji, Juntao Dai +2
Aligning the behavior of Large language models (LLMs) with human intentions and values remains a critical challenge. Reinforcement learning from human feedback (RLHF) aligns LLMs b…
SafeSora: Towards Safety Alignment of Text2Video Generation via a Human Preference Dataset
Josef Dai, Tianle Chen, Xuyao Wang +4
To mitigate the risk of harmful outputs from large vision models (LVMs), we introduce the SafeSora dataset to promote research on aligning text-to-video generation with human value…
ProgressGym: Alignment with a Millennium of Moral Progress
Tianyi Qiu, Yang Zhang, Xuchuan Huang +3
Frontier AI systems, including large language models (LLMs), hold increasing influence over the epistemology of human users. Such influence can reinforce prevailing societal values…
PKU-SafeRLHF: Towards Multi-Level Safety Alignment for LLMs with Human Preference
Jiaming Ji, Donghai Hong, Borong Zhang +10
In this study, we introduce the safety human preference dataset, PKU-SafeRLHF, designed to promote research on safety alignment in large language models (LLMs). As a sibling projec…
Language Models Resist Alignment: Evidence From Data Compression
Jiaming Ji, Kaile Wang, Tianyi Qiu +7
Large language models (LLMs) may exhibit unintended or undesirable behaviors. Recent works have concentrated on aligning LLMs to mitigate harmful outputs. Despite these efforts, so…
Reward Generalization in RLHF: A Topological Perspective
Tianyi Qiu, Fanzhi Zeng, Jiaming Ji +7
Existing alignment methods share a common topology of information flow, where reward information is collected from humans, modeled with preference learning, and used to tune langua…
Aligner: Efficient Alignment by Learning to Correct
Jiaming Ji, Boyuan Chen, Hantao Lou +6
With the rapid development of large language models (LLMs) and ever-evolving practical requirements, finding an efficient and effective alignment method has never been more critica…