Ensemble-in-One: Learning Ensemble within Random Gated Networks for Enhanced Adversarial Robustness

Abstract

Adversarial attacks have rendered high security risks on modern deep learning systems. Adversarial training can significantly enhance the robustness of neural network models by suppressing the non-robust features. However, the models often suffer from significant accuracy loss on clean data. Ensemble training methods have emerged as promising solutions for defending against adversarial attacks by diversifying the vulnerabilities among the sub-models, simultaneously maintaining comparable accuracy as standard training. However, existing ensemble methods are with poor scalability, owing to the rapid complexity increase when including more sub-models in the ensemble. Moreover, in real-world applications, it is difficult to deploy an ensemble with multiple sub-models, owing to the tight hardware resource budget and latency requirement. In this work, we propose ensemble-in-one (EIO), a simple but efficient way to train an ensemble within one random gated network (RGN). EIO augments the original model by replacing the parameterized layers with multi-path random gated blocks (RGBs) to construct a RGN. By diversifying the vulnerability of the numerous paths within the RGN, better robustness can be achieved. It provides high scalability because the paths within an EIO network exponentially increase with the network depth. Our experiments demonstrate that EIO consistently outperforms previous ensemble training methods with even less computational overhead.

References in corpus (12)

• Towards Deep Learning Models Resistant to Adversarial Attacks
• Adversarial Attacks and Defences: A Survey
• Adversarial Examples Are Not Bugs, They Are Features
• Detecting Adversarial Samples from Artifacts
• Stochastic Activation Pruning for Robust Adversarial Defense
• Mitigating Adversarial Effects Through Randomization
• Improving Adversarial Robustness via Promoting Ensemble Diversity
• Neural Network Model Extraction Attacks in Edge Devices by Hearing Architectural Hints
• Improving Adversarial Robustness of Ensembles with Diversity Training
• DVERGE: Diversifying Vulnerabilities for Enhanced Robust Generation of Ensembles
• Boosting Adversarial Attacks with Momentum
• Training Ensembles to Detect Adversarial Examples