Bypassing the Safety Training of Open-Source LLMs with Priming Attacks

Jason Vega, Isha Chaudhary, Changming Xu +1

With the recent surge in popularity of LLMs has come an ever-increasing need for LLM safety training. In this paper, we investigate the fragility of SOTA open-source LLMs under sim…