ImpNet: Imperceptible and blackbox-undetectable backdoors in compiled neural networks

Abstract

Early backdoor attacks against machine learning set off an arms race in attack and defence development. Defences have since appeared demonstrating some ability to detect backdoors in models or even remove them. These defences work by inspecting the training data, the model, or the integrity of the training procedure. In this work, we show that backdoors can be added during compilation, circumventing any safeguards in the data preparation and model training stages. The attacker can not only insert existing weight-based backdoors during compilation, but also a new class of weight-independent backdoors, such as ImpNet. These backdoors are impossible to detect during the training or data preparation processes, because they are not yet present. Next, we demonstrate that some backdoors, including ImpNet, can only be reliably detected at the stage where they are inserted and removing them anywhere else presents a significant challenge. We conclude that ML model security requires assurance of provenance along the entire technical pipeline, including the data, model architecture, compiler, and hardware specification.

arXiv comment10 pages, 7 figures, to be published in IEEE Secure and Trustworthy Machine Learning 2024. For website see https://ml.backdoors.uk . For source code, see https://sr.ht/~ecc/ImpNet

References in corpus (15)

• Explaining and Harnessing Adversarial Examples
• Intriguing properties of neural networks
• Targeted Backdoor Attacks on Deep Learning Systems Using Data Poisoning
• BadNets: Identifying Vulnerabilities in the Machine Learning Model Supply Chain
• Evasion Attacks against Machine Learning at Test Time
• BadNL: Backdoor Attacks against NLP Models with Semantic-preserving Improvements
• Relay: A New IR for Machine Learning Frameworks
• On Certifying Robustness against Backdoor Attacks via Randomized Smoothing
• Handcrafted Backdoors in Deep Neural Networks
• Backdoor Attack through Frequency Domain
• NNoculation: Catching BadNets in the Wild
• SoK: Machine Learning Governance
• Subnet Replacement: Deployment-stage backdoor attack against deep neural networks in gray-box setting
• On the Limitations of Stochastic Pre-processing Defenses
• How a simple bug in ML compiler could be exploited for backdoors?