System Prompt Extraction Attacks and Defenses in Large Language Models

Badhan Chandra Das, M. Hadi Amini, Yanzhao Wu

The system prompt in Large Language Models (LLMs) plays a pivotal role in guiding model behavior and response generation. Often containing private configuration details, user roles…