97 citations · 335 across the 14 of their papers we have counts for
5 papers · 1 filter
ImpossibleBench: Measuring LLMs' Propensity of Exploiting Test Cases
Ziqian Zhong, Aditi Raghunathan, Nicholas Carlini
The tendency to find and exploit "shortcuts" to complete tasks poses significant risks for reliable assessment and deployment of large language models (LLMs). For example, an LLM a…
Unsolved Problems in ML Safety
Dan Hendrycks, Nicholas Carlini, John Schulman +1
Machine learning (ML) systems are rapidly increasing in size, are acquiring new capabilities, and are increasingly deployed in high-stakes settings. As with other powerful technolo…
Evading Adversarial Example Detection Defenses with Orthogonal Projected Gradient Descent
Oliver Bryniarski, Nabeel Hingun, Pedro Pachuca +2
Evading adversarial example detection defenses requires finding adversarial examples that must simultaneously (a) be misclassified by the model and (b) be detected as non-adversari…
Erratum Concerning the Obfuscated Gradients Attack on Stochastic Activation Pruning
Guneet S. Dhillon, Nicholas Carlini
Stochastic Activation Pruning (SAP) (Dhillon et al., 2018) is a defense to adversarial examples that was attacked and found to be broken by the "Obfuscated Gradients" paper (Athaly…
Provably Minimally-Distorted Adversarial Examples
Nicholas Carlini, Guy Katz, Clark Barrett +1
The ability to deploy neural networks in real-world, safety-critical systems is severely limited by the presence of adversarial examples: slightly perturbed inputs that are misclas…