12 citations · 40 across the 34 of their papers we have counts for
12 papers · 1 filter
Calibrating One-Round Membership Inference with Neighbors
Francesco Rita, Jie Zhang, Florian Tramèr
The state-of-the-art Membership Inference (MI) methods calibrate their signal separately for each example using reference models, auxiliary models trained to exclude the target. Th…
Render Before Reading: Visual Rendering as a Prompt Injection Defense
Jie Zhang, Andrei Baroian, Jan N. van Rijn +2
Large language models are vulnerable to prompt injection attacks, where third-party adversarial content can hijack the model's behavior. In this paper, we study the role played by…
What Does It Mean to Break a Distillation Defense?
Lena Libon, Pura Peetathawatchai, Michael Aerni +2
Black-box LLMs (accessible only via API) are vulnerable to distillation attacks, in which an attacker queries the model and trains a student on its outputs. A recent line of work p…
Untrusted Content Masking for Web Agents with Security Guarantees
Kristina Nikolić, Egor Zverev, Javier Rando +3
Defenses that provide security guarantees against prompt injection attacks rely on strict isolation between trusted instructions and untrusted data. In text-based environments such…
Assessing Automated Prompt Injection Attacks in Agentic Environments
David Hofer, Edoardo Debenedetti, Florian Tramèr
Indirect prompt injection poses a critical threat to LLM agents that interact with untrusted external data, yet automated attack methods--proven effective for jailbreaking--remain…
Position: Anthropomorphic Misalignment Research Needs Stronger Evidence
Vansh Gupta, Peter Nutter, Samuel Stante +5
We argue that many Anthropomorphic Misalignment Research (AMR) studies need stronger evidence to ensure that they can provide a robust foundation for critical safety decisions, suc…
Laundering AI Authority with Adversarial Examples
Jie Zhang, Pura Peetathawatchai, Florian Tramèr +1
Vision-language models (VLMs) are increasingly deployed as trusted authorities -- fact-checking images on social media, comparing products, and moderating content. Users implicitly…
Reliable and Responsible Foundation Models: A Comprehensive Survey
Xinyu Yang, Junlin Han, Rishi Bommasani +49
Foundation models, including Large Language Models (LLMs), Multimodal Large Language Models (MLLMs), Image Generative Models (i.e, Text-to-Image Models and Image-Editing Models), a…
Learning to Inject: Automated Prompt Injection via Reinforcement Learning
Xin Chen, Jie Zhang, Florian Tramèr
Prompt injection is a critical vulnerability in LLM agents, yet the strongest methods still rely on human red-teamers and hand-crafted prompts. Adapting automated jailbreak optimiz…
Rethinking Benchmarks for Differentially Private Image Classification
Sabrina Mokhtari, Sara Kodeiri, Shubhankar Mohapatra +2
We revisit benchmarks for differentially private image classification. We suggest a comprehensive set of benchmarks, allowing researchers to evaluate techniques for differentially…
Representations of Text and Images Align From Layer One
Evžen Wybitul, Javier Rando, Florian Tramèr +1
We show that for a variety of concepts in adapter-based vision-language models, the representations of their images and their text descriptions are meaningfully aligned from the ve…
CaMeLs Can Use Computers Too: System-level Security for Computer Use Agents
Hanna Foerster, Tom Blanchard, Kristina Nikolić +6
AI agents are vulnerable to prompt injection attacks, where malicious content hijacks agent behavior. Among proposed defenses, architectural isolation provides the strongest guaran…