Involuntary Jailbreak: On Self-Prompting Attacks

Yangyang Guo, Yangyan Li, Mohan Kankanhalli

In this study, we disclose a worrying new vulnerability in Large Language Models (LLMs), which we term \textbf{involuntary jailbreak}. Unlike existing jailbreak attacks, this weakn…