Technical Report on the CleverHans v2.1.0 Adversarial Examples Library

Abstract

CleverHans is a software library that provides standardized reference implementations of adversarial example construction techniques and adversarial training. The library may be used to develop more robust machine learning models and to provide standardized benchmarks of models' performance in the adversarial setting. Benchmarks constructed without a standardized implementation of adversarial example construction are not comparable to each other, because a good result may indicate a robust model or it may merely indicate a weak implementation of the adversarial example construction procedure. This technical report is structured as follows. Section 1 provides an overview of adversarial examples in machine learning and of the CleverHans software. Section 2 presents the core functionalities of the library: namely the attacks based on adversarial examples and defenses to improve the robustness of machine learning models to these attacks. Section 3 describes how to report benchmark results using the library. Section 4 describes the versioning system.

arXiv commentTechnical report for https://github.com/tensorflow/cleverhans

Cited by in corpus (165)

• On Evaluating Adversarial Robustness
• Robust Physical-World Attacks on Deep Learning Models
• The Space of Transferable Adversarial Examples
• Explainable Artificial Intelligence Applications in Cyber Security: State-of-the-Art in Research
• Detecting Adversarial Samples from Artifacts
• Deep k-Nearest Neighbors: Towards Confident, Interpretable and Robust Deep Learning
• Adversarial Attacks on Deep Neural Networks for Time Series Classification
• On Adaptive Attacks to Adversarial Example Defenses
• Adversarial Examples, Uncertainty, and Transfer Testing Robustness in Gaussian Process Hybrid Deep Networks
• RobustBench: a standardized adversarial robustness benchmark
• Adversarial Attacks Against Medical Deep Learning Systems
• Discriminative Active Learning
• Machine Learning Testing: Survey, Landscapes and Horizons
• The Conditional Entropy Bottleneck
• Improving Adversarial Robustness of Ensembles with Diversity Training
• DeepRobust: A PyTorch Library for Adversarial Attacks and Defenses
• Launching Adversarial Attacks against Network Intrusion Detection Systems for IoT
• A Counter-Forensic Method for CNN-Based Camera Model Identification
• OpenAttack: An Open-source Textual Adversarial Attack Toolkit
• Ensemble Methods as a Defense to Adversarial Perturbations Against Deep Neural Networks
• Responsible and Regulatory Conform Machine Learning for Medicine: A Survey of Challenges and Solutions
• Unlabeled Data Improves Adversarial Robustness
• TextAttack: A Framework for Adversarial Attacks, Data Augmentation, and Adversarial Training in NLP
• Discretization based Solutions for Secure Machine Learning against Adversarial Attacks
• Dynamic Time Warping based Adversarial Framework for Time-Series Domain
• A Survey of Safety and Trustworthiness of Deep Neural Networks: Verification, Testing, Adversarial Attack and Defence, and Interpretability
• Adversarial Machine Learning -- Industry Perspectives
• Logit Pairing Methods Can Fool Gradient-Based Attacks
• Detecting Adversarial Samples for Deep Neural Networks through Mutation Testing
• Advbox: a toolbox to generate adversarial examples that fool neural networks
• On the Geometry of Adversarial Examples
• Saccader: Improving Accuracy of Hard Attention Models for Vision
• Whatever Does Not Kill Deep Reinforcement Learning, Makes It Stronger
• Machine Learning (In) Security: A Stream of Problems
• A Causal View on Robustness of Neural Networks
• Responsible AI by Design in Practice
• Understanding Real-world Threats to Deep Learning Models in Android Apps
• Bugs in our Pockets: The Risks of Client-Side Scanning
• Quantifying Perceptual Distortion of Adversarial Examples
• Security Matters: A Survey on Adversarial Machine Learning
• Fortified Networks: Improving the Robustness of Deep Networks by Modeling the Manifold of Hidden Representations
• Towards a Robust Deep Neural Network in Texts: A Survey
• Assaying Out-Of-Distribution Generalization in Transfer Learning
• Smoothed Geometry for Robust Attribution
• MaskDGA: A Black-box Evasion Technique Against DGA Classifiers and Adversarial Defenses
• Defending Against Adversarial Attacks by Leveraging an Entire GAN
• The RFML Ecosystem: A Look at the Unique Challenges of Applying Deep Learning to Radio Frequency Applications
• ECGadv: Generating Adversarial Electrocardiogram to Misguide Arrhythmia Classification System
• Adversarial Machine Learning: Bayesian Perspectives
• Benchmarking Adversarial Robustness
• COPYCAT: Practical Adversarial Attacks on Visualization-Based Malware Detection
• The Faults in Our Pi Stars: Security Issues and Open Challenges in Deep Reinforcement Learning
• There is Limited Correlation between Coverage and Robustness for Deep Neural Networks
• Adversarial Training Versus Weight Decay
• Improving adversarial robustness of deep neural networks by using semantic information
• MULDEF: Multi-model-based Defense Against Adversarial Examples for Neural Networks
• Poison as a Cure: Detecting & Neutralizing Variable-Sized Backdoor Attacks in Deep Neural Networks
• Adversarial Training with Voronoi Constraints
• FenceBox: A Platform for Defeating Adversarial Examples with Data Augmentation Techniques
• The Search for Sparse, Robust Neural Networks
• Adversarially Robust Training through Structured Gradient Regularization
• Adversarial Framework with Certified Robustness for Time-Series Domain via Statistical Features
• Mitigating Advanced Adversarial Attacks with More Advanced Gradient Obfuscation Techniques
• AllenNLP Interpret: A Framework for Explaining Predictions of NLP Models
• Adversarial Attacks for Tabular Data: Application to Fraud Detection and Imbalanced Data
• Adversarial Attacks and Defense on Texts: A Survey
• Two Souls in an Adversarial Image: Towards Universal Adversarial Example Detection using Multi-view Inconsistency
• When Attackers Meet AI: Learning-empowered Attacks in Cooperative Spectrum Sensing
• Featurized Bidirectional GAN: Adversarial Defense via Adversarially Learned Semantic Inference
• Detecting Adversarial Examples and Other Misclassifications in Neural Networks by Introspection
• Enhancing Gradient-based Attacks with Symbolic Intervals
• Subverting Fair Image Search with Generative Adversarial Perturbations
• ATHENA: A Framework based on Diverse Weak Defenses for Building Adversarial Defense
• PuVAE: A Variational Autoencoder to Purify Adversarial Examples
• Predictive Uncertainty Quantification with Compound Density Networks
• secml: A Python Library for Secure and Explainable Machine Learning
• Realistic Adversarial Data Augmentation for MR Image Segmentation
• Towards Robust Deep Neural Networks
• A Marauder's Map of Security and Privacy in Machine Learning
• Building Robust Deep Neural Networks for Road Sign Detection
• Deep learning for time series classification
• Invert and Defend: Model-based Approximate Inversion of Generative Adversarial Networks for Secure Inference
• Attribution-driven Causal Analysis for Detection of Adversarial Examples
• Adversarial Machine Learning Phases of Matter
• Identifying Classes Susceptible to Adversarial Attacks
• Improved Adversarial Robustness via Logit Regularization Methods
• Gradient Similarity: An Explainable Approach to Detect Adversarial Attacks against Deep Learning
• A critique of the DeepSec Platform for Security Analysis of Deep Learning Models
• Using Single-Step Adversarial Training to Defend Iterative Adversarial Examples
• Effects of Loss Functions And Target Representations on Adversarial Robustness
• RecRec: Algorithmic Recourse for Recommender Systems
• Stochastic Substitute Training: A Gray-box Approach to Craft Adversarial Examples Against Gradient Obfuscation Defenses
• Towards an Awareness of Time Series Anomaly Detection Models' Adversarial Vulnerability
• Detecting Localized Adversarial Examples: A Generic Approach using Critical Region Analysis
• A Comprehensive Evaluation Framework for Deep Model Robustness
• Towards Rapid and Robust Adversarial Training with One-Step Attacks
• Adversarial Transferability in Wearable Sensor Systems
• Random Directional Attack for Fooling Deep Neural Networks
• The Efficacy of SHIELD under Different Threat Models
• GanDef: A GAN based Adversarial Training Defense for Neural Network Classifier
• Subset Scanning Over Neural Network Activations
• Adversarial Examples in Constrained Domains
• AutoGAN: Robust Classifier Against Adversarial Attacks
• Structure-Preserving Transformation: Generating Diverse and Transferable Adversarial Examples
• Attacking CNN-based anti-spoofing face authentication in the physical domain
• Towards an Understanding of Neural Networks in Natural-Image Spaces
• Minimax Defense against Gradient-based Adversarial Attacks
• Purifying Adversarial Perturbation with Adversarially Trained Auto-encoders
• Adversarial Defense Through Network Profiling Based Path Extraction
• ASP:A Fast Adversarial Attack Example Generation Framework based on Adversarial Saliency Prediction
• Defending Adversarial Attacks by Correcting logits
• Perturbations are not Enough: Generating Adversarial Examples with Spatial Distortions
• TESDA: Transform Enabled Statistical Detection of Attacks in Deep Neural Networks
• Blind Adversarial Training: Balance Accuracy and Robustness
• ROBY: Evaluating the Robustness of a Deep Model by its Decision Boundaries
• Latent Adversarial Defence with Boundary-guided Generation
• New CleverHans Feature: Better Adversarial Robustness Evaluations with Attack Bundling
• Patch augmentation: Towards efficient decision boundaries for neural networks
• Perceptually Constrained Adversarial Attacks
• What it Thinks is Important is Important: Robustness Transfers through Input Gradients
• Guiding the retraining of convolutional neural networks against adversarial inputs
• The Care Label Concept: A Certification Suite for Trustworthy and Resource-Aware Machine Learning
• ZK-GanDef: A GAN based Zero Knowledge Adversarial Training Defense for Neural Networks
• ExAD: An Ensemble Approach for Explanation-based Adversarial Detection
• Generating Adversarial Examples with an Optimized Quality
• Building Reliable Explanations of Unreliable Neural Networks: Locally Smoothing Perspective of Model Interpretation
• MixDefense: A Defense-in-Depth Framework for Adversarial Example Detection Based on Statistical and Semantic Analysis
• Feedback Learning for Improving the Robustness of Neural Networks
• FoveaTer: Foveated Transformer for Image Classification
• The Human Visual System and Adversarial AI
• Robustifying automatic speech recognition by extracting slowly varying features
• Attack Type Agnostic Perceptual Enhancement of Adversarial Images
• When Side-Channel Attacks Break the Black-Box Property of Embedded Artificial Intelligence
• Detecting Adversarial Samples Using Density Ratio Estimates
• Adversarial Machine Learning Attacks on Condition-Based Maintenance Capabilities
• Deep Repulsive Prototypes for Adversarial Robustness
• The best defense is a good offense: Countering black box attacks by predicting slightly wrong labels
• Unifying Bilateral Filtering and Adversarial Training for Robust Neural Networks
• Can the state of relevant neurons in a deep neural networks serve as indicators for detecting adversarial attacks?
• On Lyapunov exponents and adversarial perturbation
• Blind Adversarial Pruning: Balance Accuracy, Efficiency and Robustness
• Target Training Does Adversarial Training Without Adversarial Samples
• Law and Adversarial Machine Learning
• TEAM: We Need More Powerful Adversarial Examples for DNNs
• Training Machine Learning Models by Regularizing their Explanations
• Enhancing Transformation-based Defenses using a Distribution Classifier
• Learning to Disentangle Robust and Vulnerable Features for Adversarial Detection
• Tricking Adversarial Attacks To Fail
• Verifying the Causes of Adversarial Examples
• Query-Free Adversarial Transfer via Undertrained Surrogates
• Visualizing Representations of Adversarially Perturbed Inputs
• MadNet: Using a MAD Optimization for Defending Against Adversarial Attacks
• PDPGD: Primal-Dual Proximal Gradient Descent Adversarial Attack
• Robust Deep Neural Networks Inspired by Fuzzy Logic
• OOWL500: Overcoming Dataset Collection Bias in the Wild
• On Connections between Regularizations for Improving DNN Robustness
• Towards Leveraging the Information of Gradients in Optimization-based Adversarial Attack
• Adversarial Examples as an Input-Fault Tolerance Problem
• Measuring the Contribution of Multiple Model Representations in Detecting Adversarial Instances
• FUNN: Flexible Unsupervised Neural Network
• Attribution of Gradient Based Adversarial Attacks for Reverse Engineering of Deceptions
• Investigating maximum likelihood based training of infinite mixtures for uncertainty quantification
• Deep Detector Health Management under Adversarial Campaigns
• TEAM: An Taylor Expansion-Based Method for Generating Adversarial Examples
• Adversarial Attack Attribution: Discovering Attributable Signals in Adversarial ML Attacks