11 papers
SafePyramid: A Hierarchical Benchmark for In-context Policy Guardrailing
Jiacheng Zhang, Haoyu He, Sen Zhang +5
In real-world applications, guardrails are often expected to identify unsafe user-model interactions according to application-specific safety policies, rather than relying on prede…
USAD: Uncertainty-aware Statistical Adversarial Detection
Zhijian Zhou, Xunye Tian, Jiacheng Zhang +5
Statistical adversarial detection (SAD) treats detection as a two-sample test. Given a reference set of clean examples (CEs) and a batch of queries, potentially containing an unkno…
From Risk Classification to Action Plan Remediation: A Guardrail Feedback Driven Framework for LLM Agents
Yuhao Sun, Jiacheng Zhang, Shaanan Cohney +3
LLM-based guardrails typically safeguard agents by evaluating proposed actions or inputs before execution, producing safety signals such as binary allow/deny decisions, risk catego…
Multi-Turn Adaptive Prompting Attack on Large Vision-Language Models
In Chong Choi, Jiacheng Zhang, Feng Liu +1
Multi-turn jailbreak attacks have proven effective against text-only large language models (LLMs), where malicious content is gradually introduced to bypass safety alignment. Howev…
SAVAA: Mitigating Hallucinations in LVLMs via Step-wise Adaptive Visual Attention Amplification
Jiacheng Zhang, Feng Liu, Chao Du +1
A line of recent training-free methods for mitigating hallucinations in large vision-language models (LVLMs) operates by amplifying attention to visual tokens during autoregressive…
Semantic-aware Adversarial Fine-tuning for CLIP
Jiacheng Zhang, Jinhao Li, Hanxun Huang +3
Recent studies have shown that CLIP model's adversarial robustness in zero-shot classification tasks can be enhanced by adversarially fine-tuning its image encoder with adversarial…