Meta Approach to Data Augmentation Optimization
arXiv:2006.07965
Abstract
Data augmentation policies drastically improve the performance of image recognition tasks, especially when the policies are optimized for the target data and tasks. In this paper, we propose to optimize image recognition models and data augmentation policies simultaneously to improve the performance using gradient descent. Unlike prior methods, our approach avoids using proxy tasks or reducing search space, and can directly improve the validation performance. Our method achieves efficient and scalable training by approximating the gradient of policies by implicit gradient with Neumann series approximation. We demonstrate that our approach can improve the performance of various image classification tasks, including ImageNet classification and fine-grained recognition, without using dataset-specific hyperparameter tuning.
References in corpus (11)
- PyTorch: An Imperative Style, High-Performance Deep Learning Library
- A Simple Framework for Contrastive Learning of Visual Representations
- mixup: Beyond Empirical Risk Minimization
- Improved Regularization of Convolutional Neural Networks with Cutout
- Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computation
- SGDR: Stochastic Gradient Descent with Warm Restarts
- Momentum Contrast for Unsupervised Visual Representation Learning
- AutoAugment: Learning Augmentation Policies from Data
- Gradient-based Hyperparameter Optimization through Reversible Learning
- Hyperparameter optimization with approximate gradient
- Improving Robustness Without Sacrificing Accuracy with Patch Gaussian Augmentation