Online Hyper-parameter Learning for Auto-Augmentation Strategy
arXiv:1905.07373
Abstract
Data augmentation is critical to the success of modern deep learning techniques. In this paper, we propose Online Hyper-parameter Learning for Auto-Augmentation (OHL-Auto-Aug), an economical solution that learns the augmentation policy distribution along with network training. Unlike previous methods on auto-augmentation that search augmentation strategies in an offline manner, our method formulates the augmentation policy as a parameterized probability distribution, thus allowing its parameters to be optimized jointly with network parameters. Our proposed OHL-Auto-Aug eliminates the need of re-training and dramatically reduces the cost of the overall search process, while establishes significantly accuracy improvements over baseline models. On both CIFAR-10 and ImageNet, our method achieves remarkable on search accuracy, 60x faster on CIFAR-10 and 24x faster on ImageNet, while maintaining competitive accuracies.
References in corpus (19)
- Practical Bayesian Optimization of Machine Learning Algorithms
- mixup: Beyond Empirical Risk Minimization
- Neural Architecture Search with Reinforcement Learning
- Improved Regularization of Convolutional Neural Networks with Cutout
- The Effectiveness of Data Augmentation in Image Classification using Deep Learning
- Accurate, Large Minibatch SGD: Training ImageNet in 1 Hour
- Wide Residual Networks
- Data Augmentation Generative Adversarial Networks
- AutoAugment: Learning Augmentation Policies from Data
- Efficient Neural Architecture Search via Parameter Sharing
- Neural Architecture Optimization
- Smart Augmentation - Learning an Optimal Data Augmentation Strategy
- Deep Image: Scaling up Image Recognition
- Convolutional Neural Fabrics
- A Bayesian Data Augmentation Approach for Learning Deep Models
- Data Augmentation in Emotion Classification Using Generative Adversarial Networks
- Dual Path Networks
- Critical Hyper-Parameters: No Random, No Cry
- IRLAS: Inverse Reinforcement Learning for Architecture Search