On the security relevance of weights in deep learning

Abstract

Recently, a weight-based attack on stochastic gradient descent inducing overfitting has been proposed. We show that the threat is broader: A task-independent permutation on the initial weights suffices to limit the achieved accuracy to for example 50% on the Fashion MNIST dataset from initially more than $90$%. These findings are confirmed on MNIST and CIFAR. We formally confirm that the attack succeeds with high likelihood and does not depend on the data. Empirically, weight statistics and loss appear unsuspicious, making it hard to detect the attack if the user is not aware. Our paper is thus a call for action to acknowledge the importance of the initial weights in deep learning.

arXiv comment16 pages, 18 figures, long version of paper published at ICANN 2020

References in corpus (9)

• On the difficulty of training Recurrent Neural Networks
• Wild Patterns: Ten Years After the Rise of Adversarial Machine Learning
• Poisoning Attacks against Support Vector Machines
• Exponential expressivity in deep neural networks through transient chaos
• Deep Neural Networks with Random Gaussian Weights: A Universal Classification Strategy?
• Thou Shalt Not Depend on Me: Analysing the Use of Outdated JavaScript Libraries on the Web
• On the Robustness of Convolutional Neural Networks to Internal Architecture and Weight Perturbations
• Bad Global Minima Exist and SGD Can Reach Them
• A Study of Deep Learning Robustness Against Computation Failures