Adversarial AutoAugment

Abstract

Data augmentation (DA) has been widely utilized to improve generalization in training deep neural networks. Recently, human-designed data augmentation has been gradually replaced by automatically learned augmentation policy. Through finding the best policy in well-designed search space of data augmentation, AutoAugment can significantly improve validation accuracy on image classification tasks. However, this approach is not computationally practical for large-scale problems. In this paper, we develop an adversarial method to arrive at a computationally-affordable solution called Adversarial AutoAugment, which can simultaneously optimize target related object and augmentation policy search loss. The augmentation policy network attempts to increase the training loss of a target network through generating adversarial augmentation policies, while the target network can learn more robust features from harder examples to improve the generalization. In contrast to prior work, we reuse the computation in target network training for policy evaluation, and dispense with the retraining of the target network. Compared to AutoAugment, this leads to about 12x reduction in computing cost and 11x shortening in time overhead on ImageNet. We show experimental results of our approach on CIFAR-10/CIFAR-100, ImageNet, and demonstrate significant performance improvements over state-of-the-art. On CIFAR-10, we achieve a top-1 test error of 1.36%, which is the currently best performing single model. On ImageNet, we achieve a leading performance of top-1 accuracy 79.40% on ResNet-50 and 80.00% on ResNet-50-D without extra data.

ICLR2020

References in corpus (1)

• Neural Architecture Search with Reinforcement Learning

Cited by in corpus (20)

• AutoML: A Survey of the State-of-the-Art
• A Comprehensive Survey of Image Augmentation Techniques for Deep Learning
• Automated data processing and feature engineering for deep learning and big data applications: a survey
• Image Augmentations for GAN Training
• AADG: Automatic Augmentation for Domain Generalization on Retinal Image Segmentation
• An overview of mixing augmentation methods and augmentation strategies
• Image quality assessment for machine learning tasks using meta-reinforcement learning
• UniformAugment: A Search-free Probabilistic Data Augmentation Approach
• Self-Supervised Neuron Segmentation with Multi-Agent Reinforcement Learning
• Neural Transformation Learning for Deep Anomaly Detection Beyond Images
• PointAugment: an Auto-Augmentation Framework for Point Cloud Classification
• KeepAugment: A Simple Information-Preserving Data Augmentation Approach
• A Second look at Exponential and Cosine Step Sizes: Simplicity, Adaptivity, and Performance
• Image quality assessment by overlapping task-specific and task-agnostic measures: application to prostate multiparametric MR images for cancer segmentation
• Improving the Transferability of Adversarial Attacks on Face Recognition with Beneficial Perturbation Feature Augmentation
• Collaboration of Experts: Achieving 80% Top-1 Accuracy on ImageNet with 100M FLOPs
• Joint Search of Data Augmentation Policies and Network Architectures
• Robust and Accurate Object Detection via Adversarial Learning
• MetaAugment: Sample-Aware Data Augmentation Policy Learning
• LA3: Efficient Label-Aware AutoAugment