Dual Head Adversarial Training

Abstract

Deep neural networks (DNNs) are known to be vulnerable to adversarial examples/attacks, raising concerns about their reliability in safety-critical applications. A number of defense methods have been proposed to train robust DNNs resistant to adversarial attacks, among which adversarial training has so far demonstrated the most promising results. However, recent studies have shown that there exists an inherent tradeoff between accuracy and robustness in adversarially-trained DNNs. In this paper, we propose a novel technique Dual Head Adversarial Training (DH-AT) to further improve the robustness of existing adversarial training methods. Different from existing improved variants of adversarial training, DH-AT modifies both the architecture of the network and the training strategy to seek more robustness. Specifically, DH-AT first attaches a second network head (or branch) to one intermediate layer of the network, then uses a lightweight convolutional neural network (CNN) to aggregate the outputs of the two heads. The training strategy is also adapted to reflect the relative importance of the two heads. We empirically show, on multiple benchmark datasets, that DH-AT can bring notable robustness improvements to existing adversarial training methods. Compared with TRADES, one state-of-the-art adversarial training method, our DH-AT can improve the robustness by 3.4% against PGD40 and 2.3% against AutoAttack, and also improve the clean accuracy by 1.8%.

References in corpus (18)

• Deep Speech 2: End-to-End Speech Recognition in English and Mandarin
• Feature Squeezing: Detecting Adversarial Examples in Deep Neural Networks
• Wide Residual Networks
• Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attacks
• Understanding Adversarial Attacks on Deep Learning Based Medical Image Analysis Systems
• Improving the Adversarial Robustness and Interpretability of Deep Neural Networks by Regularizing their Input Gradients
• On the Convergence and Robustness of Adversarial Training
• Skip Connections Matter: On the Transferability of Adversarial Examples Generated with ResNets
• Sparse DNNs with Improved Adversarial Robustness
• Adv-BERT: BERT is not robust on misspellings! Generating nature adversarial samples on BERT
• Boosting Adversarial Training with Hypersphere Embedding
• Unravelling Robustness of Deep Learning based Face Recognition Against Adversarial Attacks
• Adversarial Objects Against LiDAR-Based Autonomous Driving Systems
• Improving Adversarial Robustness via Channel-wise Activation Suppressing
• Do Wider Neural Networks Really Help Adversarial Robustness?
• Learnable Boundary Guided Adversarial Training
• Bag of Tricks for Adversarial Training
• On the Limitations of Denoising Strategies as Adversarial Defenses