7 papers
Oyster-II: Reinforcement Learning for Constructive Safety Alignment in Large Language Models
Jiyang Guan, Yong Xie, Jun Chen +6
Large language models (LLMs) have demonstrated remarkable capabilities across diverse applications, yet ensuring their simultaneous safety, helpfulness, and trustworthiness remains…
Inverse Reinforcement Learning with Dynamic Reward Scaling for LLM Alignment
Ruoxi Cheng, Haoxuan Ma, Weixin Wang +7
Alignment is vital for safely deploying large language models (LLMs). Existing techniques are either reward-based (training a reward model on preference pairs and optimizing with r…
A Causal Perspective for Enhancing Jailbreak Attack and Defense
Licheng Pan, Yunsheng Lu, Jiexi Liu +5
Uncovering the mechanisms behind "jailbreaks" in large language models (LLMs) is crucial for enhancing their safety and reliability, yet these mechanisms remain poorly understood.…
MoAPT: Mixture of Adversarial Prompt Tuning for Vision-Language Models
Shiji Zhao, Qihui Zhu, Shukun Xiong +7
Large pre-trained Vision Language Models (VLMs) demonstrate excellent generalization capabilities but remain highly susceptible to adversarial examples, posing potential security r…
Oyster-I: Beyond Refusal -- Constructive Safety Alignment for Responsible Language Models
Ranjie Duan, Jiexi Liu, Xiaojun Jia +27
Large language models (LLMs) typically deploy safety mechanisms to prevent harmful content generation. Most current approaches focus narrowly on risks posed by malicious actors, of…
Strata-Sword: A Hierarchical Safety Evaluation towards LLMs based on Reasoning Complexity of Jailbreak Instructions
Shiji Zhao, Ranjie Duan, Jiexi Liu +10
Large language models (LLMs) have gained widespread recognition for their superior comprehension and have been deployed across numerous domains. Building on Chain-of-Thought (CoT)…