A Neuro-Inspired Autoencoding Defense Against Adversarial Perturbations

Abstract

Deep Neural Networks (DNNs) are vulnerable to adversarial attacks: carefully constructed perturbations to an image can seriously impair classification accuracy, while being imperceptible to humans. While there has been a significant amount of research on defending against such attacks, most defenses based on systematic design principles have been defeated by appropriately modified attacks. For a fixed set of data, the most effective current defense is to train the network using adversarially perturbed examples. In this paper, we investigate a radically different, neuro-inspired defense mechanism, starting from the observation that human vision is virtually unaffected by adversarial examples designed for machines. We aim to reject L^inf bounded adversarial perturbations before they reach a classifier DNN, using an encoder with characteristics commonly observed in biological vision: sparse overcomplete representations, randomness due to synaptic noise, and drastic nonlinearities. Encoder training is unsupervised, using standard dictionary learning. A CNN-based decoder restores the size of the encoder output to that of the original image, enabling the use of a standard CNN for classification. Our nominal design is to train the decoder and classifier together in standard supervised fashion, but we also consider unsupervised decoder training based on a regression objective (as in a conventional autoencoder) with separate supervised training of the classifier. Unlike adversarial training, all training is based on clean images. Our experiments on the CIFAR-10 show performance competitive with state-of-the-art defenses based on adversarial training, and point to the promise of neuro-inspired techniques for the design of robust neural networks. In addition, we provide results for a subset of the Imagenet dataset to verify that our approach scales to larger images.

References in corpus (14)

• PyTorch: An Imperative Style, High-Performance Deep Learning Library
• EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks
• Transferability in Machine Learning: from Phenomena to Black-Box Attacks using Adversarial Samples
• Theoretically Principled Trade-off between Robustness and Accuracy
• Synthesizing Robust Adversarial Examples
• On Evaluating Adversarial Robustness
• Certified Defenses against Adversarial Examples
• Adversarially Robust Generalization Requires More Data
• Countering Adversarial Images using Input Transformations
• Stochastic Activation Pruning for Robust Adversarial Defense
• k-Sparse Autoencoders
• On Adaptive Attacks to Adversarial Example Defenses
• ME-Net: Towards Effective Adversarial Robustness with Matrix Estimation
• Robust Adversarial Learning via Sparsifying Front Ends