Self-supervised Neural Architecture Search

Abstract

Neural Architecture Search (NAS) has been used recently to achieve improved performance in various tasks and most prominently in image classification. Yet, current search strategies rely on large labeled datasets, which limit their usage in the case where only a smaller fraction of the data is annotated. Self-supervised learning has shown great promise in training neural networks using unlabeled data. In this work, we propose a self-supervised neural architecture search (SSNAS) that allows finding novel network models without the need for labeled data. We show that such a search leads to comparable results to supervised training with a "fully labeled" NAS and that it can improve the performance of self-supervised learning. Moreover, we demonstrate the advantage of the proposed approach when the number of labels in the search is relatively small.

References in corpus (11)

• A Simple Framework for Contrastive Learning of Visual Representations
• Improved Baselines with Momentum Contrastive Learning
• DARTS: Differentiable Architecture Search
• AutoAugment: Learning Augmentation Policies from Data
• Contrastive Multiview Coding
• Large-Scale Evolution of Image Classifiers
• Robust Large Margin Deep Neural Networks
• Self-training with Noisy Student improves ImageNet classification
• Fixing the train-test resolution discrepancy: FixEfficientNet
• ASAP: Architecture Search, Anneal and Prune
• AGAN: Towards Automated Design of Generative Adversarial Networks

Cited by in corpus (5)

• Contrastive Self-supervised Neural Architecture Search
• Weight-Sharing Neural Architecture Search: A Battle to Shrink the Optimization Gap
• Self-supervised Cross-silo Federated Neural Architecture Search
• Self-Supervised Neural Architecture Search for Imbalanced Datasets
• Self-Supervised Neural Architecture Search for Multimodal Deep Neural Networks