1.1k citations · 3k across the 8 of their papers we have counts for
9 papers
A Cheap Verifier is Good Enough: LLM Post-training is Robust to Erroneous Rewards
Andreas Plesner, Curtis Northcutt, Francisco Guzmán +1
When post-training large language models on tasks with semi-verifiable rewards, there are many factors (training steps, base model size, training order, data quality, verifier accu…
An Imperfect Verifier is Good Enough: Learning with Noisy Rewards
Andreas Plesner, Francisco Guzmán, Anish Athalye
Reinforcement Learning with Verifiable Rewards (RLVR) is widely used for post-training Large Language Models, but practical verifiers can make errors. We study how the rate and str…
Pervasive Label Errors in Test Sets Destabilize Machine Learning Benchmarks
Curtis G. Northcutt, Anish Athalye, Jonas Mueller
We identify label errors in the test sets of 10 of the most commonly-used computer vision, natural language, and audio datasets, and subsequently study the potential for these labe…
On Evaluating Adversarial Robustness
Nicholas Carlini, Anish Athalye, Nicolas Papernot +6
Correctly evaluating defenses against adversarial examples has proven to be extremely difficult. Despite the significant amount of recent work attempting to design defenses that wi…
Evaluating and Understanding the Robustness of Adversarial Logit Pairing
Logan Engstrom, Andrew Ilyas, Anish Athalye
We evaluate the robustness of Adversarial Logit Pairing, a recently proposed defense against adversarial examples. We find that a network trained with Adversarial Logit Pairing ach…
On the Robustness of the CVPR 2018 White-Box Adversarial Example Defenses
Anish Athalye, Nicholas Carlini
Neural networks are known to be vulnerable to adversarial examples. In this note, we evaluate the two white-box defenses that appeared at CVPR 2018 and find they are ineffective: w…
Black-box Adversarial Attacks with Limited Queries and Information
Andrew Ilyas, Logan Engstrom, Anish Athalye +1
Current neural network-based classifiers are susceptible to adversarial examples even in the black-box setting, where the attacker only has query access to the model. In practice,…
Obfuscated Gradients Give a False Sense of Security: Circumventing Defenses to Adversarial Examples
Anish Athalye, Nicholas Carlini, David Wagner
We identify obfuscated gradients, a kind of gradient masking, as a phenomenon that leads to a false sense of security in defenses against adversarial examples. While defenses that…
Synthesizing Robust Adversarial Examples
Anish Athalye, Logan Engstrom, Andrew Ilyas +1
Standard methods for generating adversarial examples for neural networks do not consistently fool neural network classifiers in the physical world due to a combination of viewpoint…