Improving Auto-Augment via Augmentation-Wise Weight Sharing

Abstract

The recent progress on automatically searching augmentation policies has boosted the performance substantially for various tasks. A key component of automatic augmentation search is the evaluation process for a particular augmentation policy, which is utilized to return reward and usually runs thousands of times. A plain evaluation process, which includes full model training and validation, would be time-consuming. To achieve efficiency, many choose to sacrifice evaluation reliability for speed. In this paper, we dive into the dynamics of augmented training of the model. This inspires us to design a powerful and efficient proxy task based on the Augmentation-Wise Weight Sharing (AWS) to form a fast yet accurate evaluation process in an elegant way. Comprehensive analysis verifies the superiority of this approach in terms of effectiveness and efficiency. The augmentation policies found by our method achieve superior accuracies compared with existing auto-augmentation search methods. On CIFAR-10, we achieve a top-1 error rate of 1.24%, which is currently the best performing single model without extra training data. On ImageNet, we get a top-1 error rate of 20.36% for ResNet-50, which leads to 3.34% absolute error rate reduction over the baseline augmentation.

arXiv commentAccepted to NeurIPS 2020 (Poster)

References in corpus (14)

• Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
• mixup: Beyond Empirical Risk Minimization
• Neural Architecture Search with Reinforcement Learning
• DeepPose: Human Pose Estimation via Deep Neural Networks
• Improved Regularization of Convolutional Neural Networks with Cutout
• Wide Residual Networks
• DARTS: Differentiable Architecture Search
• Neural Architecture Optimization
• Data Augmentation by Pairing Samples for Images Classification
• Deep Image: Scaling up Image Recognition
• Shake-Shake regularization
• Convolutional Neural Fabrics
• Population Based Augmentation: Efficient Learning of Augmentation Policy Schedules
• Powering One-shot Topological NAS with Stabilized Share-parameter Proxy

Cited by in corpus (5)

• Automated data processing and feature engineering for deep learning and big data applications: a survey
• Advancements in Point Cloud Data Augmentation for Deep Learning: A Survey
• Sample-aware RandAugment: Search-free Automatic Data Augmentation for Effective Image Recognition
• Direct Differentiable Augmentation Search
• LA3: Efficient Label-Aware AutoAugment