21 papers
Agent Against Agent: An Agentic System for Automatic Prompt Injection Red Teaming
Yanting Wang, Chenlong Yin, Runpeng Geng +1
Prompt injection poses significant security risks to LLM agents. Efficient and effective red-teaming is therefore critical, both for evaluating these risks and for collecting train…
FlashRT: Towards Computationally and Memory Efficient Red-Teaming for Prompt Injection and Knowledge Corruption
Yanting Wang, Chenlong Yin, Ying Chen +1
Long-context large language models (LLMs)-for example, Gemini-3.1-Pro and Qwen-3.5-are widely used to empower many real-world applications, such as retrieval-augmented generation,…
PIArena: A Platform for Prompt Injection Evaluation
Runpeng Geng, Chenlong Yin, Yanting Wang +2
Prompt injection attacks pose serious security risks across a wide range of real-world applications. While receiving increasing attention, the community faces a critical gap: the l…
AgentWatcher: A Rule-based Prompt Injection Monitor
Yanting Wang, Wei Zou, Runpeng Geng +1
Large language models (LLMs) and their applications, such as agents, are highly vulnerable to prompt injection attacks. State-of-the-art prompt injection detection methods have the…
EnsembleSHAP: Faithful and Certifiably Robust Attribution for Random Subspace Method
Yanting Wang, Jinyuan Jia
Random subspace method has wide security applications such as providing certified defenses against adversarial and backdoor attacks, and building robustly aligned LLM against jailb…
PISmith: Reinforcement Learning-based Red Teaming for Prompt Injection Defenses
Chenlong Yin, Runpeng Geng, Yanting Wang +1
Prompt injection poses serious security risks to real-world LLM applications, particularly autonomous agents. Although many defenses have been proposed, their robustness against ad…
Be Your Own Red Teamer: Safety Alignment via Self-Play and Reflective Experience Replay
Hao Wang, Yanting Wang, Hao Li +2
Large Language Models (LLMs) have achieved remarkable capabilities but remain vulnerable to adversarial ``jailbreak'' attacks designed to bypass safety guardrails. Current safety a…
Native Parallel Reasoner: Reasoning in Parallelism via Self-Distilled Reinforcement Learning
Tong Wu, Yang Liu, Jun Bai +6
We introduce Native Parallel Reasoner (NPR), a teacher-free framework that enables Large Language Models (LLMs) to self-evolve genuine parallel reasoning capabilities. NPR transfor…
TASO: Jailbreak LLMs via Alternative Template and Suffix Optimization
Yanting Wang, Runpeng Geng, Jinghui Chen +2
Many recent studies showed that LLMs are vulnerable to jailbreak attacks, where an attacker can perturb the input of an LLM to induce it to generate an output for a harmful questio…
ATLAS: A High-Difficulty, Multidisciplinary Benchmark for Frontier Scientific Reasoning
Hongwei Liu, Junnan Liu, Shudong Liu +33
The rapid advancement of Large Language Models (LLMs) has led to performance saturation on many established benchmarks, questioning their ability to distinguish frontier models. Co…
PISanitizer: Preventing Prompt Injection to Long-Context LLMs via Prompt Sanitization
Runpeng Geng, Yanting Wang, Chenlong Yin +3
Long context LLMs are vulnerable to prompt injection, where an attacker can inject an instruction in a long context to induce an LLM to generate an attacker-desired output. Existin…
PIShield: Detecting Prompt Injection Attacks via Intrinsic LLM Features
Wei Zou, Yupei Liu, Yanting Wang +3
LLM-integrated applications are vulnerable to prompt injection attacks, where an attacker contaminates the input to inject malicious instructions, causing the LLM to follow the att…
SecInfer: Preventing Prompt Injection via Inference-time Scaling
Yupei Liu, Yanting Wang, Yuqi Jia +2
Prompt injection attacks pose a pervasive threat to the security of Large Language Models (LLMs). State-of-the-art prevention-based defenses typically rely on fine-tuning an LLM to…
UniC-RAG: Universal Knowledge Corruption Attacks to Retrieval-Augmented Generation
Runpeng Geng, Yanting Wang, Ying Chen +1
Retrieval-augmented generation (RAG) systems are widely deployed in real-world applications in diverse domains such as finance, healthcare, and cybersecurity. However, many studies…
MCPTox: A Benchmark for Tool Poisoning Attack on Real-World MCP Servers
Zhiqiang Wang, Yichao Gao, Yanting Wang +6
By providing a standardized interface for LLM agents to interact with external tools, the Model Context Protocol (MCP) is quickly becoming a cornerstone of the modern autonomous ag…
AttnTrace: Contextual Attribution of Prompt Injection and Knowledge Corruption
Yanting Wang, Runpeng Geng, Ying Chen +1
Long-context large language models (LLMs), such as Gemini-2.5-Pro and Claude-Sonnet-4, are increasingly used to empower advanced AI systems, including retrieval-augmented generatio…
TracLLM: A Generic Framework for Attributing Long Context LLMs
Yanting Wang, Wei Zou, Runpeng Geng +1
Long context large language models (LLMs) are deployed in many real-world applications such as RAG, agent, and broad LLM-integrated applications. Given an instruction and a long co…
TrojanDec: Data-free Detection of Trojan Inputs in Self-supervised Learning
Yupei Liu, Yanting Wang, Jinyuan Jia
An image encoder pre-trained by self-supervised learning can be used as a general-purpose feature extractor to build downstream classifiers for various downstream tasks. However, m…
TrojFM: Resource-efficient Backdoor Attacks against Very Large Foundation Models
Yuzhou. Nie, Yanting. Wang, Jinyuan. Jia +4
One key challenge in backdoor attacks against large foundation models is the resource limits. Backdoor attacks usually require retraining the target model, which is impractical for…
FCert: Certifiably Robust Few-Shot Classification in the Era of Foundation Models
Yanting Wang, Wei Zou, Jinyuan Jia
Few-shot classification with foundation models (e.g., CLIP, DINOv2, PaLM-2) enables users to build an accurate classifier with a few labeled training samples (called support sample…
MMCert: Provable Defense against Adversarial Attacks to Multi-modal Models
Yanting Wang, Hongye Fu, Wei Zou +1
Different from a unimodal model whose input is from a single modality, the input (called multi-modal input) of a multi-modal model is from multiple modalities such as image, 3D poi…