Attacking Binarized Neural Networks

Abstract

Neural networks with low-precision weights and activations offer compelling efficiency advantages over their full-precision equivalents. The two most frequently discussed benefits of quantization are reduced memory consumption, and a faster forward pass when implemented with efficient bitwise operations. We propose a third benefit of very low-precision neural networks: improved robustness against some adversarial attacks, and in the worst case, performance that is on par with full-precision models. We focus on the very low-precision case where weights and activations are both quantized to $\pm$1, and note that stochastically quantizing weights in just one layer can sharply reduce the impact of iterative attacks. We observe that non-scaled binary neural networks exhibit a similar effect to the original defensive distillation procedure that led to gradient masking, and a false notion of security. We address this by conducting both black-box and white-box experiments with binary models that do not artificially mask gradients.

Published as a conference paper at ICLR 2018

References in corpus (6)

• Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
• Binarized Neural Networks: Training Deep Neural Networks with Weights and Activations Constrained to +1 or -1
• Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computation
• DoReFa-Net: Training Low Bitwidth Convolutional Neural Networks with Low Bitwidth Gradients
• Towards Deep Learning Models Resistant to Adversarial Attacks
• Adversarial Machine Learning at Scale

Cited by in corpus (14)

• Threat of Adversarial Attacks on Deep Learning in Computer Vision: A Survey
• Adversarial Sample Detection for Deep Neural Network through Model Mutation Testing
• Detecting Adversarial Samples for Deep Neural Networks through Mutation Testing
• Defensive Quantization: When Efficiency Meets Robustness
• DeepHunter: Hunting Deep Neural Network Defects via Coverage-Guided Fuzzing
• Exploring Adversarial Attack in Spiking Neural Networks with Spike-Compatible Gradient
• DarKnight: A Data Privacy Scheme for Training and Inference of Deep Neural Networks
• Adversarial Examples on Object Recognition: A Comprehensive Survey
• Improving Adversarial Robustness in Weight-quantized Neural Networks
• Accelerating Binarized Neural Networks via Bit-Tensor-Cores in Turing GPUs
• BCNN: Binary Complex Neural Network
• Robust binary classification with the 01 loss
• On the transferability of adversarial examples between convex and 01 loss models
• Towards adversarial robustness with 01 loss neural networks