37 citations · 63 across the 25 of their papers we have counts for
27 papers
Repeat-After-Me: Black-Box Adaptive Visual Prompt Injection
Sizhe Chen, Yu-Lin Tsai, Ivan Evtimov +4
Prompt injection is widely recognized as a major security threat to AI agents that interact with untrusted external data, such as websites, documents, and emails. Prior work has sh…
Inadvertent Context Leakage in Language Models
Jaiden Fairoze, Neal Mangaokar, Kamalika Chaudhuri +2
For AI agents to be useful beyond simple chat, they must hold sensitive user context such as calendars, credentials, health records, and financial data. We study whether the mere p…
Muse Spark Safety & Preparedness Report
Cristina Menghini, Peter Ney, Hamza Kwisaba +117
Muse Spark is the latest large language model developed by Meta. In this report, we first present evaluations for catastrophic risk domains under Meta's Advanced AI Scaling Framewo…
Privacy Blur: Quantifying Privacy and Utility for Image Data Release
Saeed Mahloujifar, Narine Kokhlikyan, Chuan Guo +1
Image data collected in the wild often contains private information such as faces and license plates, and responsible data release must ensure that this information stays hidden. A…
Safety Alignment of LMs via Non-cooperative Games
Anselm Paulus, Ilia Kulikov, Brandon Amos +4
Ensuring the safety of language models (LMs) while maintaining their usefulness remains a critical challenge in AI alignment. Current approaches rely on sequential adversarial trai…
CIMemories: A Compositional Benchmark for Contextual Integrity of Persistent Memory in LLMs
Niloofar Mireshghallah, Neal Mangaokar, Narine Kokhlikyan +4
Large Language Models (LLMs) increasingly use persistent memory from past interactions to enhance personalization and task performance. However, this memory introduces critical ris…
Z0-Inf: Zeroth Order Approximation for Data Influence
Narine Kokhlikyan, Kamalika Chaudhuri, Saeed Mahloujifar
A critical aspect of analyzing and improving modern machine learning systems lies in understanding how individual training examples influence a model's predictive behavior. Estimat…
RL Is a Hammer and LLMs Are Nails: A Simple Reinforcement Learning Recipe for Strong Prompt Injection
Yuxin Wen, Arman Zharmagambetov, Ivan Evtimov +4
Prompt injection poses a serious threat to the reliability and safety of LLM agents. Recent defenses against prompt injection, such as Instruction Hierarchy and SecAlign, have show…
Machine Learning with Privacy for Protected Attributes
Saeed Mahloujifar, Chuan Guo, G. Edward Suh +1
Differential privacy (DP) has become the standard for private data analysis. Certain machine learning applications only require privacy protection for specific protected attributes…
How much do language models memorize?
John X. Morris, Chawin Sitawarin, Chuan Guo +5
We propose a new method for estimating how much a model knows about a datapoint and use it to measure the capacity of modern language models. Prior studies of language model memori…
AbstentionBench: Reasoning LLMs Fail on Unanswerable Questions
Polina Kirichenko, Mark Ibrahim, Kamalika Chaudhuri +1
For Large Language Models (LLMs) to be reliably deployed in both everyday and high-stakes domains, knowing when not to answer is equally critical as answering correctly. Real-world…
WASP: Benchmarking Web Agent Security Against Prompt Injection Attacks
Ivan Evtimov, Arman Zharmagambetov, Aaron Grattafiori +2
Autonomous UI agents powered by AI have tremendous potential to boost human productivity by automating routine tasks such as filing taxes and paying bills. However, a major challen…
Measuring Déjà vu Memorization Efficiently
Narine Kokhlikyan, Bargav Jayaraman, Florian Bordes +2
Recent research has shown that representation learning models may accidentally memorize their training data. For example, the déjà vu method shows that for certain representation l…
AgentDAM: Privacy Leakage Evaluation for Autonomous Web Agents
Arman Zharmagambetov, Chuan Guo, Ivan Evtimov +3
Autonomous AI agents that can follow instructions and perform complex multi-step tasks have tremendous potential to boost human productivity. However, to perform many of these task…
Auditing -Differential Privacy in One Run
Saeed Mahloujifar, Luca Melis, Kamalika Chaudhuri
Empirical auditing has emerged as a means of catching some of the flaws in the implementation of privacy-preserving algorithms. Existing auditing mechanisms, however, are either co…
SecAlign: Defending Against Prompt Injection with Preference Optimization
Sizhe Chen, Arman Zharmagambetov, Saeed Mahloujifar +3
Large language models (LLMs) are becoming increasingly prevalent in modern software systems, interfacing between the user and the Internet to assist with tasks that require advance…
An Introduction to Vision-Language Modeling
Florian Bordes, Richard Yuanzhe Pang, Anurag Ajay +38
Following the recent popularity of Large Language Models (LLMs), several attempts have been made to extend them to the visual domain. From having a visual assistant that could guid…
Uncertainty-Based Abstention in LLMs Improves Safety and Reduces Hallucinations
Christian Tomani, Kamalika Chaudhuri, Ivan Evtimov +2
A major barrier towards the practical deployment of large language models (LLMs) is their lack of reliability. Three situations where this is particularly apparent are correctness,…
Guarantees of confidentiality via Hammersley-Chapman-Robbins bounds
Kamalika Chaudhuri, Chuan Guo, Laurens van der Maaten +2
Protecting privacy during inference with deep neural networks is possible by adding noise to the activations in the last layers prior to the final classifiers or other task-specifi…
Privacy Amplification for the Gaussian Mechanism via Bounded Support
Shengyuan Hu, Saeed Mahloujifar, Virginia Smith +2
Data-dependent privacy accounting frameworks such as per-instance differential privacy (pDP) and Fisher information loss (FIL) confer fine-grained privacy guarantees for individual…
DP-RDM: Adapting Diffusion Models to Private Domains Without Fine-Tuning
Jonathan Lebensold, Maziar Sanjabi, Pietro Astolfi +4
Text-to-image diffusion models have been shown to suffer from sample-level memorization, possibly reproducing near-perfect replica of images that they are trained on, which may be…
Differentially Private Representation Learning via Image Captioning
Tom Sander, Yaodong Yu, Maziar Sanjabi +4
Differentially private (DP) machine learning is considered the gold-standard solution for training a model from sensitive data while still preserving privacy. However, a major barr…
Measuring Privacy Loss in Distributed Spatio-Temporal Data
Tatsuki Koga, Casey Meehan, Kamalika Chaudhuri
Statistics about traffic flow and people's movement gathered from multiple geographical locations in a distributed manner are the driving force powering many applications, such as…
Déjà Vu Memorization in Vision-Language Models
Bargav Jayaraman, Chuan Guo, Kamalika Chaudhuri
Vision-Language Models (VLMs) have emerged as the state-of-the-art representation learning solution, with myriads of downstream applications such as image classification, retrieval…