Unrestricted Adversarial Examples

Abstract

We introduce a two-player contest for evaluating the safety and robustness of machine learning systems, with a large prize pool. Unlike most prior work in ML robustness, which studies norm-constrained adversaries, we shift our focus to unconstrained adversaries. Defenders submit machine learning models, and try to achieve high accuracy and coverage on non-adversarial data while making no confident mistakes on adversarial inputs. Attackers try to subvert defenses by finding arbitrary unambiguous inputs where the model assigns an incorrect label with high confidence. We propose a simple unambiguous dataset ("bird-or- bicycle") to use as part of this contest. We hope this contest will help to more comprehensively evaluate the worst-case adversarial risk of machine learning models.

References in corpus (4)

• Intriguing properties of neural networks
• Playing Atari with Deep Reinforcement Learning
• On Calibration of Modern Neural Networks
• Obfuscated Gradients Give a False Sense of Security: Circumventing Defenses to Adversarial Examples

Cited by in corpus (38)

• Theoretically Principled Trade-off between Robustness and Accuracy
• Natural Adversarial Examples
• Constructing Unrestricted Adversarial Examples with Generative Models
• Scalable agent alignment via reward modeling: a research direction
• RobustBench: a standardized adversarial robustness benchmark
• Adversarial Training against Location-Optimized Adversarial Patches
• Unrestricted Adversarial Examples via Semantic Manipulation
• Excessive Invariance Causes Adversarial Vulnerability
• Rigorous Agent Evaluation: An Adversarial Approach to Uncover Catastrophic Failures
• SemanticAdv: Generating Adversarial Examples via Attribute-conditional Image Editing
• Perceptual Adversarial Robustness: Defense Against Unseen Threat Models
• Breaking certified defenses: Semantic adversarial examples with spoofed robustness certificates
• Improving Robustness of Task Oriented Dialog Systems
• Random Smoothing Might be Unable to Certify ${\ell }_{\infty }$ Robustness for High-Dimensional Images
• Adversarial Color Enhancement: Generating Unrestricted Adversarial Images by Optimizing a Color Filter
• LTD: Low Temperature Distillation for Gradient Masking-free Adversarial Training
• Adversarial Image Translation: Unrestricted Adversarial Examples in Face Recognition Systems
• Towards Large yet Imperceptible Adversarial Image Perturbations with Perceptual Color Distance
• Adaptive Generation of Unrestricted Adversarial Inputs
• Fine-grained Synthesis of Unrestricted Adversarial Examples
• Security and Machine Learning in the Real World
• Adversarial Attacks and Defenses: An Interpretation Perspective
• Built-in Vulnerabilities to Imperceptible Adversarial Perturbations
• Identifying Model Weakness with Adversarial Examiner
• Jointly Optimizing Preprocessing and Inference for DNN-based Visual Analytics
• Tiny Adversarial Mulit-Objective Oneshot Neural Architecture Search
• On Norm-Agnostic Robustness of Adversarial Training
• Understanding the Error in Evaluating Adversarial Robustness
• Developing Creative AI to Generate Sculptural Objects
• An Adversarially-Learned Turing Test for Dialog Generation Models
• Collaborative Sampling in Generative Adversarial Networks
• Generating Unrestricted Adversarial Examples via Three Parameters
• Blind Adversarial Pruning: Balance Accuracy, Efficiency and Robustness
• Domain Invariant Adversarial Learning
• Regulatory Markets for AI Safety
• Towards Robustness against Unsuspicious Adversarial Examples
• Adversarial Examples as an Input-Fault Tolerance Problem
• Hallucinating Point Cloud into 3D Sculptural Object