5 papers
Climbing the Hill: Prompt Injection Red-Teaming Against Frontier Models with Curriculum Reinforcement Learning
Chenlong Yin, Xiaolong Jin, Wei Zou +2
Prompt injection is a leading security risk for LLMs and LLM-based applications such as agents. State-of-the-art red-teaming methods for prompt injection leverage reinforcement lea…
When Do Intrinsic Rewards Work for Code Reasoning? A Comprehensive Study
Xiaolong Jin, Xuandong Zhao, Wenbo Guo +2
Reinforcement learning with verifiable rewards (RLVR) has driven substantial progress in large language model reasoning, but relies on ground-truth supervision that is costly or in…
ASTRA: Autonomous Spatial-Temporal Red-teaming for AI Software Assistants
Xiangzhe Xu, Guangyu Shen, Zian Su +9
AI coding assistants like GitHub Copilot are rapidly transforming software development, but their safety remains deeply uncertain-especially in high-stakes domains like cybersecuri…
Foot-In-The-Door: A Multi-turn Jailbreak for LLMs
Zixuan Weng, Xiaolong Jin, Jinyuan Jia +1
Ensuring AI safety is crucial as large language models become increasingly integrated into real-world applications. A key challenge is jailbreak, where adversarial prompts bypass b…
MULTIVERSE: Exposing Large Language Model Alignment Problems in Diverse Worlds
Xiaolong Jin, Zhuo Zhang, Xiangyu Zhang
Large Language Model (LLM) alignment aims to ensure that LLM outputs match with human values. Researchers have demonstrated the severity of alignment problems with a large spectrum…