1.6k citations · 2.2k across the 52 of their papers we have counts for
12 papers · 1 filter
Rebellion: Noise-Robust Reasoning Training for Audio Reasoning Models
Tiansheng Huang, Virat Shejwalkar, Oscar Chang +2
Instilling reasoning capabilities in large models (LMs) using reasoning training (RT) significantly improves LMs' performances. Thus Audio Reasoning Models (ARMs), i.e., audio LMs…
SIRAJ: Diverse and Efficient Red-Teaming for LLM Agents via Distilled Structured Reasoning
Kaiwen Zhou, Ahmed Elgohary, A S M Iftekhar +1
The ability of LLM agents to plan and invoke tools exposes them to new safety risks, making a comprehensive red-teaming system crucial for discovering vulnerabilities and ensuring…
The Attacker Moves Second: Stronger Adaptive Attacks Bypass Defenses Against Llm Jailbreaks and Prompt Injections
Milad Nasr, Nicholas Carlini, Chawin Sitawarin +11
How should we evaluate the robustness of language model defenses? Current defenses against jailbreaks and prompt injections (which aim to prevent an attacker from eliciting harmful…
Evaluating the Robustness of a Production Malware Detection System to Transferable Adversarial Attacks
Milad Nasr, Yanick Fratantonio, Luca Invernizzi +7
As deep learning models become widely deployed as components within larger production systems, their individual shortcomings can create system-level vulnerabilities with real-world…
Hush! Protecting Secrets During Model Training: An Indistinguishability Approach
Arun Ganesh, Brendan McMahan, Milad Nasr +2
We consider the problem of secret protection, in which a business or organization wishes to train a model on their own data, while attempting to not leak secrets potentially contai…
Lessons from Defending Gemini Against Indirect Prompt Injections
Chongyang Shi, Sharon Lin, Shuang Song +11
Gemini is increasingly used to perform tasks on behalf of users, where function-calling and tool-use capabilities enable the model to access user data. Some tools, however, require…
LLMs unlock new paths to monetizing exploits
Nicholas Carlini, Milad Nasr, Edoardo Debenedetti +5
We argue that Large language models (LLMs) will soon alter the economics of cyberattacks. Instead of attacking the most commonly used software and monetizing exploits by targeting…
Cascading Adversarial Bias from Injection to Distillation in Language Models
Harsh Chaudhari, Jamie Hayes, Matthew Jagielski +3
Model distillation has become essential for creating smaller, deployable language models that retain larger system capabilities. However, widespread deployment raises concerns abou…
Exploring the limits of strong membership inference attacks on large language models
Jamie Hayes, Ilia Shumailov, Christopher A. Choquette-Choo +13
State-of-the-art membership inference attacks (MIAs) typically require training many reference models, making it difficult to scale these attacks to large pre-trained language mode…
Privacy Auditing of Large Language Models
Ashwinee Panda, Xinyu Tang, Milad Nasr +2
Current techniques for privacy auditing of large language models (LLMs) have limited efficacy -- they rely on basic approaches to generate canaries which leads to weak membership i…
AutoAdvExBench: Benchmarking autonomous exploitation of adversarial example defenses
Nicholas Carlini, Javier Rando, Edoardo Debenedetti +2
We introduce AutoAdvExBench, a benchmark to evaluate if large language models (LLMs) can autonomously exploit defenses to adversarial examples. Unlike existing security benchmarks…
Exploring and Mitigating Adversarial Manipulation of Voting-Based Leaderboards
Yangsibo Huang, Milad Nasr, Anastasios Angelopoulos +10
It is now common to evaluate Large Language Models (LLMs) by having humans manually vote to evaluate model outputs, in contrast to typical benchmarks that evaluate knowledge or ski…