Population Based Augmentation: Efficient Learning of Augmentation Policy Schedules

Abstract

A key challenge in leveraging data augmentation for neural network training is choosing an effective augmentation policy from a large search space of candidate operations. Properly chosen augmentation policies can lead to significant generalization improvements; however, state-of-the-art approaches such as AutoAugment are computationally infeasible to run for the ordinary user. In this paper, we introduce a new data augmentation algorithm, Population Based Augmentation (PBA), which generates nonstationary augmentation policy schedules instead of a fixed augmentation policy. We show that PBA can match the performance of AutoAugment on CIFAR-10, CIFAR-100, and SVHN, with three orders of magnitude less overall compute. On CIFAR-10 we achieve a mean test error of 1.46%, which is a slight improvement upon the current state-of-the-art. The code for PBA is open source and is available at https://github.com/arcelien/pba.

ICML 2019

Cited by in corpus (51)

• Albumentations: fast and flexible image augmentations
• AutoML: A Survey of the State-of-the-Art
• Selective Synthetic Augmentation with HistoGAN for Improved Histopathology Image Classification
• Data Augmenting Contrastive Learning of Speech Representations in the Time Domain
• An overview of mixing augmentation methods and augmentation strategies
• Feature Fusion from Head to Tail for Long-Tailed Visual Recognition
• Model Patching: Closing the Subgroup Performance Gap with Data Augmentation
• Greedy Policy Search: A Simple Baseline for Learnable Test-Time Augmentation
• DADA: Differentiable Automatic Data Augmentation
• ResizeMix: Mixing Data with Preserved Object Information and True Labels
• Data Augmentation Revisited: Rethinking the Distribution Gap between Clean and Augmented Data
• Improving Auto-Augment via Augmentation-Wise Weight Sharing
• Optimized Generic Feature Learning for Few-shot Classification across Domains
• Weight-Sharing Neural Architecture Search: A Battle to Shrink the Optimization Gap
• Viewmaker Networks: Learning Views for Unsupervised Representation Learning
• Neural Transformation Learning for Deep Anomaly Detection Beyond Images
• Visual Imitation Made Easy
• Adversarial Reinforced Instruction Attacker for Robust Vision-Language Navigation
• PointAugment: an Auto-Augmentation Framework for Point Cloud Classification
• Role-Wise Data Augmentation for Knowledge Distillation
• EnAET: A Self-Trained framework for Semi-Supervised and Supervised Learning with Ensemble Transformations
• WeMix: How to Better Utilize Data Augmentation
• Improving 3D Object Detection through Progressive Population Based Augmentation
• Learning by Analogy: Reliable Supervision from Transformations for Unsupervised Optical Flow Estimation
• SCARF: Self-Supervised Contrastive Learning using Random Feature Corruption
• Circumventing Outliers of AutoAugment with Knowledge Distillation
• Learn to Augment: Joint Data Augmentation and Network Optimization for Text Recognition
• Scale-aware Automatic Augmentation for Object Detection
• Rethinking Recurrent Neural Networks and Other Improvements for Image Classification
• VEGA: Towards an End-to-End Configurable AutoML Pipeline
• Learning Test-time Augmentation for Content-based Image Retrieval
• A Self-Supervised Feature Map Augmentation (FMA) Loss and Combined Augmentations Finetuning to Efficiently Improve the Robustness of CNNs
• DeltaGAN: Towards Diverse Few-shot Image Generation with Sample-Specific Delta
• Selective Synthetic Augmentation with Quality Assurance
• Tuning Mixed Input Hyperparameters on the Fly for Efficient Population Based AutoRL
• Stage-based Hyper-parameter Optimization for Deep Learning
• StackMix: A complementary Mix algorithm
• Hippo: Taming Hyper-parameter Optimization of Deep Learning with Stage Trees
• Self-Supervised Domain Adaptation with Consistency Training
• Hypernetwork-Based Augmentation
• Local Patch AutoAugment with Multi-Agent Collaboration
• Mixup Without Hesitation
• ParamCrop: Parametric Cubic Cropping for Video Contrastive Learning
• RPCL: A Framework for Improving Cross-Domain Detection with Auxiliary Tasks
• DeepMix: Online Auto Data Augmentation for Robust Visual Object Tracking
• MetaAugment: Sample-Aware Data Augmentation Policy Learning
• How Data Augmentation affects Optimization for Linear Regression
• Learning Identity-Preserving Transformations on Data Manifolds
• In-Loop Meta-Learning with Gradient-Alignment Reward
• Fine-Grained AutoAugmentation for Multi-Label Classification
• Text AutoAugment: Learning Compositional Augmentation Policy for Text Classification