Prompt Packer: Deceiving LLMs through Compositional Instruction with Hidden Attacks

Shuyu Jiang, Xingshu Chen, Rui Tang

Recently, Large language models (LLMs) with powerful general capabilities have been increasingly integrated into various Web applications, while undergoing alignment training to en…