54 citations · 74 across the 15 of their papers we have counts for
16 papers
GPT-Red: Automated Red Teaming via Self-Play at Scale
Eric Wallace, Christopher A. Choquette-Choo, Nikhil Kandpal +15
We introduce \textbf{GPT-Red}, an automated red-teaming agent that is trained to discover novel prompt injection attacks against frontier LLMs. The goal of this model is to evaluat…
IH-Challenge: A Training Dataset to Improve Instruction Hierarchy on Frontier LLMs
Chuan Guo, Juan Felipe Ceron Uribe, Sicheng Zhu +10
Instruction hierarchy (IH) defines how LLMs prioritize system, developer, user, and tool instructions under conflict, providing a concrete, trust-ordered policy for resolving instr…
MIST-RL: Mutation-based Incremental Suite Testing via Reinforcement Learning
Sicheng Zhu, Jiajun Wang, Jiawei Ai +1
Large Language Models (LLMs) often fail to generate correct code on the first attempt, which requires using generated unit tests as verifiers to validate the solutions. Despite the…
Like Oil and Water: Group Robustness Methods and Poisoning Defenses May Be at Odds
Michael-Andrei Panaitescu-Liess, Yigitcan Kaya, Sicheng Zhu +2
Group robustness has become a major concern in machine learning (ML) as conventional training paradigms were found to produce high error on minority groups. Without explicit group…
PoisonedParrot: Subtle Data Poisoning Attacks to Elicit Copyright-Infringing Content from Large Language Models
Michael-Andrei Panaitescu-Liess, Pankayaraj Pathmanathan, Yigitcan Kaya +5
As the capabilities of large language models (LLMs) continue to expand, their usage has become increasingly prevalent. However, as reflected in numerous ongoing lawsuits regarding…
AdvPrefix: An Objective for Nuanced LLM Jailbreaks
Sicheng Zhu, Brandon Amos, Yuandong Tian +2
Many jailbreak attacks on large language models (LLMs) rely on a common objective: making the model respond with the prefix ``Sure, here is (harmful request)''. While straightforwa…
GenARM: Reward Guided Generation with Autoregressive Reward Model for Test-time Alignment
Yuancheng Xu, Udari Madhushani Sehwag, Alec Koppel +4
Large Language Models (LLMs) exhibit impressive capabilities but require careful alignment with human preferences. Traditional training-time methods finetune LLMs using human prefe…
Automatic Pseudo-Harmful Prompt Generation for Evaluating False Refusals in Large Language Models
Bang An, Sicheng Zhu, Ruiyi Zhang +3
Safety-aligned large language models (LLMs) sometimes falsely refuse pseudo-harmful prompts, like "how to kill a mosquito," which are actually harmless. Frequent false refusals not…
Can Watermarking Large Language Models Prevent Copyrighted Text Generation and Hide Training Data?
Michael-Andrei Panaitescu-Liess, Zora Che, Bang An +6
Large Language Models (LLMs) have demonstrated impressive capabilities in generating diverse and contextually rich text. However, concerns regarding copyright infringement arise as…
WAVES: Benchmarking the Robustness of Image Watermarks
Bang An, Mucong Ding, Tahseen Rabbani +8
In the burgeoning age of generative AI, watermarks act as identifiers of provenance and artificial content. We present WAVES (Watermark Analysis Via Enhanced Stress-testing), a ben…
AutoDAN: Interpretable Gradient-Based Adversarial Attacks on Large Language Models
Sicheng Zhu, Ruiyi Zhang, Bang An +6
Safety alignment of Large Language Models (LLMs) can be compromised with manual jailbreak attacks and (automatic) adversarial attacks. Recent studies suggest that defending against…
PerceptionCLIP: Visual Classification by Inferring and Conditioning on Contexts
Bang An, Sicheng Zhu, Michael-Andrei Panaitescu-Liess +2
Vision-language models like CLIP are widely used in zero-shot image classification due to their ability to understand various visual concepts and natural language descriptions. How…
On the Possibilities of AI-Generated Text Detection
Souradip Chakraborty, Amrit Singh Bedi, Sicheng Zhu +3
Our work addresses the critical issue of distinguishing text generated by Large Language Models (LLMs) from human-produced text, a task essential for numerous applications. Despite…
Understanding the Generalization Benefit of Model Invariance from a Data Perspective
Sicheng Zhu, Bang An, Furong Huang
Machine learning models that are developed with invariance to certain types of data transformations have demonstrated superior generalization performance in practice. However, the…
Learning Adversarially Robust Representations via Worst-Case Mutual Information Maximization
Sicheng Zhu, Xiao Zhang, David Evans
Training machine learning models that are robust against adversarial inputs poses seemingly insurmountable challenges. To better understand adversarial robustness, we consider the…
Guess First to Enable Better Compression and Adversarial Robustness
Sicheng Zhu, Bang An, Shiyu Niu
Machine learning models are generally vulnerable to adversarial examples, which is in contrast to the robustness of humans. In this paper, we try to leverage one of the mechanisms…