Adversarial Self-Supervised Contrastive Learning

Abstract

Existing adversarial learning approaches mostly use class labels to generate adversarial samples that lead to incorrect predictions, which are then used to augment the training of the model for improved robustness. While some recent works propose semi-supervised adversarial learning methods that utilize unlabeled data, they still require class labels. However, do we really need class labels at all, for adversarially robust training of deep neural networks? In this paper, we propose a novel adversarial attack for unlabeled data, which makes the model confuse the instance-level identities of the perturbed data samples. Further, we present a self-supervised contrastive learning framework to adversarially train a robust neural network without labeled data, which aims to maximize the similarity between a random augmentation of a data sample and its instance-wise adversarial perturbation. We validate our method, Robust Contrastive Learning (RoCL), on multiple benchmark datasets, on which it obtains comparable robust accuracy over state-of-the-art supervised adversarial learning methods, and significantly improved robustness against the black box and unseen types of attacks. Moreover, with further joint fine-tuning with supervised adversarial loss, RoCL obtains even higher robust accuracy over using self-supervised learning alone. Notably, RoCL also demonstrate impressive results in robust transfer learning.

arXiv commentNeurIPS 2020. Code: https://github.com/Kim-Minseon/RoCL

References in corpus (13)

• Accurate, Large Minibatch SGD: Training ImageNet in 1 Hour
• Wide Residual Networks
• SGDR: Stochastic Gradient Descent with Warm Restarts
• Theoretically Principled Trade-off between Robustness and Accuracy
• What Makes for Good Views for Contrastive Learning?
• Certified Adversarial Robustness via Randomized Smoothing
• Large Batch Training of Convolutional Networks
• Using Pre-Training Can Improve Model Robustness and Uncertainty
• Provably Robust Deep Learning via Adversarially Trained Smoothed Classifiers
• Certified Adversarial Robustness with Additive Noise
• Are Labels Required for Improving Adversarial Robustness?
• Revisiting Self-Supervised Visual Representation Learning
• Self-supervised Adversarial Training

Cited by in corpus (8)

• Empowering the 6G Cellular Architecture with Open RAN
• Uncertainty-aware Contrastive Distillation for Incremental Semantic Segmentation
• Self-Paced Contrastive Learning for Semi-supervised Medical Image Segmentation with Meta-labels
• Robust Proxy: Improving Adversarial Robustness by Robust Proxy Learning
• Towards Deviation-Robust Agent Navigation via Perturbation-Aware Contrastive Learning
• Deep Contrastive Graph Representation via Adaptive Homotopy Learning
• Adversarial Training for Face Recognition Systems using Contrastive Adversarial Learning and Triplet Loss Fine-tuning
• THAT: Two Head Adversarial Training for Improving Robustness at Scale