13 papers
Agent Against Agent: An Agentic System for Automatic Prompt Injection Red Teaming
Yanting Wang, Chenlong Yin, Runpeng Geng +1
Prompt injection poses significant security risks to LLM agents. Efficient and effective red-teaming is therefore critical, both for evaluating these risks and for collecting train…
PIArena: A Platform for Prompt Injection Evaluation
Runpeng Geng, Chenlong Yin, Yanting Wang +2
Prompt injection attacks pose serious security risks across a wide range of real-world applications. While receiving increasing attention, the community faces a critical gap: the l…
TRUSTDESC: Preventing Tool Poisoning in LLM Applications via Trusted Description Generation
Hengkai Ye, Zhechang Zhang, Jinyuan Jia +1
Large language models (LLMs) increasingly rely on external tools to perform time-sensitive tasks and real-world actions. While tool integration expands LLM capabilities, it also in…
Evaluating LLM-based Personal Information Extraction and Countermeasures
Yupei Liu, Yuqi Jia, Jinyuan Jia +1
Automatically extracting personal information -- such as name, phone number, and email address -- from publicly available profiles at a large scale is a stepstone to many other sec…
EnsembleSHAP: Faithful and Certifiably Robust Attribution for Random Subspace Method
Yanting Wang, Jinyuan Jia
Random subspace method has wide security applications such as providing certified defenses against adversarial and backdoor attacks, and building robustly aligned LLM against jailb…
PISmith: Reinforcement Learning-based Red Teaming for Prompt Injection Defenses
Chenlong Yin, Runpeng Geng, Yanting Wang +1
Prompt injection poses serious security risks to real-world LLM applications, particularly autonomous agents. Although many defenses have been proposed, their robustness against ad…