Blocking Transferability of Adversarial Examples in Black-Box Learning Systems

Abstract

Advances in Machine Learning (ML) have led to its adoption as an integral component in many applications, including banking, medical diagnosis, and driverless cars. To further broaden the use of ML models, cloud-based services offered by Microsoft, Amazon, Google, and others have developed ML-as-a-service tools as black-box systems. However, ML classifiers are vulnerable to adversarial examples: inputs that are maliciously modified can cause the classifier to provide adversary-desired outputs. Moreover, it is known that adversarial examples generated on one classifier are likely to cause another classifier to make the same mistake, even if the classifiers have different architectures or are trained on disjoint datasets. This property, which is known as transferability, opens up the possibility of attacking black-box systems by generating adversarial examples on a substitute classifier and transferring the examples to the target classifier. Therefore, the key to protect black-box learning systems against the adversarial examples is to block their transferability. To this end, we propose a training method that, as the input is more perturbed, the classifier smoothly outputs lower confidence on the original label and instead predicts that the input is "invalid". In essence, we augment the output class set with a NULL label and train the classifier to reject the adversarial examples by classifying them as NULL. In experiments, we apply a wide range of attacks based on adversarial examples on the black-box systems. We show that a classifier trained with the proposed method effectively resists against the adversarial examples, while maintaining the accuracy on clean data.

References in corpus (17)

• Transferability in Machine Learning: from Phenomena to Black-Box Attacks using Adversarial Samples
• Towards Deep Neural Network Architectures Robust to Adversarial Examples
• Understanding Adversarial Training: Increasing Local Stability of Neural Nets through Robust Optimization
• Adversarial Perturbations Against Deep Neural Networks for Malware Classification
• Learning with a Strong Adversary
• A study of the effect of JPG compression on adversarial images
• Deceiving Google's Perspective API Built for Detecting Toxic Comments
• Practical Black-Box Attacks against Machine Learning
• Universal adversarial perturbations
• Adversarial Images for Variational Autoencoders
• Adversarial examples for generative models
• Suppressing the Unusual: towards Robust CNNs using Symmetric Activation Functions
• Assessing Threat of Adversarial Examples on Deep Neural Networks
• Improving Back-Propagation by Adding an Adversarial Gradient
• Using Non-invertible Data Transformations to Build Adversarial-Robust Neural Networks
• Adversary Resistant Deep Neural Networks with an Application to Malware Detection
• Invariant backpropagation: how to train a transformation-invariant neural network

Cited by in corpus (25)

• Adversarial Attacks and Defences: A Survey
• Adversarial Example Detection for DNN Models: A Review and Experimental Comparison
• Threat of Adversarial Attacks on Deep Learning in Computer Vision: A Survey
• Turning Your Weakness Into a Strength: Watermarking Deep Neural Networks by Backdooring
• Extending Defensive Distillation
• Invisible Mask: Practical Attacks on Face Recognition with Infrared
• Adversarial Examples in Modern Machine Learning: A Review
• Adversarial Attacks and Defenses in Physiological Computing: A Systematic Review
• A Survey of Black-Box Adversarial Attacks on Computer Vision Models
• Google's Cloud Vision API Is Not Robust To Noise
• Trustworthy AI in the Age of Pervasive Computing and Big Data
• A Target-Agnostic Attack on Deep Models: Exploiting Security Vulnerabilities of Transfer Learning
• Distortion Agnostic Deep Watermarking
• Adversarial Reinforcement Learning under Partial Observability in Autonomous Computer Network Defence
• Exploiting Vulnerabilities of Load Forecasting Through Adversarial Attacks
• Butterfly Effect: Bidirectional Control of Classification Performance by Small Additive Perturbation
• Weight Map Layer for Noise and Adversarial Attack Robustness
• Mitigating Gradient-based Adversarial Attacks via Denoising and Compression
• On The Relationship Between Universal Adversarial Attacks And Sparse Representations
• Is Machine Learning in Power Systems Vulnerable?
• Reinforcement Learning for Autonomous Defence in Software-Defined Networking
• Beyond Categorical Label Representations for Image Classification
• Deep Bayesian Image Set Classification: A Defence Approach against Adversarial Attacks
• Detecting Adversaries, yet Faltering to Noise? Leveraging Conditional Variational AutoEncoders for Adversary Detection in the Presence of Noisy Images
• Oriole: Thwarting Privacy against Trustworthy Deep Learning Models