Sockpuppetting: Jailbreaking LLMs by Combining Prefilling with Optimization

Asen Dotsinski, Panagiotis Eustratiadis

Prefill attacks are an effective and low-cost jailbreaking method, as they directly insert an acceptance sequence (e.g., "Sure, here is...") at the start of an LLM's output and lea…