CutMix: Regularization Strategy to Train Strong Classifiers with Localizable Features

Abstract

Regional dropout strategies have been proposed to enhance the performance of convolutional neural network classifiers. They have proved to be effective for guiding the model to attend on less discriminative parts of objects (e.g. leg as opposed to head of a person), thereby letting the network generalize better and have better object localization capabilities. On the other hand, current methods for regional dropout remove informative pixels on training images by overlaying a patch of either black pixels or random noise. Such removal is not desirable because it leads to information loss and inefficiency during training. We therefore propose the CutMix augmentation strategy: patches are cut and pasted among training images where the ground truth labels are also mixed proportionally to the area of the patches. By making efficient use of training pixels and retaining the regularization effect of regional dropout, CutMix consistently outperforms the state-of-the-art augmentation strategies on CIFAR and ImageNet classification tasks, as well as on the ImageNet weakly-supervised localization task. Moreover, unlike previous augmentation methods, our CutMix-trained ImageNet classifier, when used as a pretrained model, results in consistent performance gains in Pascal detection and MS-COCO image captioning benchmarks. We also show that CutMix improves the model robustness against input corruptions and its out-of-distribution detection performances. Source code and pretrained models are available at https://github.com/clovaai/CutMix-PyTorch .

Accepted at ICCV 2019 (oral talk). 14 pages, 5 figures

References in corpus (14)

• Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
• Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
• Two-Stream Convolutional Networks for Action Recognition in Videos
• mixup: Beyond Empirical Risk Minimization
• Improved Regularization of Convolutional Neural Networks with Cutout
• A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networks
• LSUN: Construction of a Large-scale Image Dataset using Deep Learning with Humans in the Loop
• A Simple Unified Framework for Detecting Out-of-Distribution Samples and Adversarial Attacks
• Random Erasing Data Augmentation
• DropBlock: A regularization method for convolutional networks
• Gather-Excite: Exploiting Feature Context in Convolutional Neural Networks
• Manifold Mixup: Better Representations by Interpolating Hidden States
• ShakeDrop Regularization for Deep Residual Learning
• NSML: Meet the MLaaS platform with a real-world case study

Cited by in corpus (64)

• Albumentations: fast and flexible image augmentations
• Transformer in Transformer
• AugMix: A Simple Data Processing Method to Improve Robustness and Uncertainty
• Masked Autoencoders Are Scalable Vision Learners
• You Only Look at One Sequence: Rethinking Transformer in Vision through Object Detection
• Deep learning for detection and segmentation of artefact and disease instances in gastrointestinal endoscopy
• DPT: Deformable Patch-based Transformer for Visual Recognition
• Fixing Data Augmentation to Improve Adversarial Robustness
• FMix: Enhancing Mixed Sample Data Augmentation
• DSP: Dual Soft-Paste for Unsupervised Domain Adaptive Semantic Segmentation
• Remix: Rebalanced Mixup
• i-Mix: A Domain-Agnostic Strategy for Contrastive Representation Learning
• Feature Fusion from Head to Tail for Long-Tailed Visual Recognition
• Self-Augmentation: Generalizing Deep Networks to Unseen Classes for Few-Shot Learning
• Unlearnable Examples: Making Personal Data Unexploitable
• Deep Weakly-Supervised Learning Methods for Classification and Localization in Histology Images: A Survey
• Domain Generalization with MixStyle
• An Empirical Evaluation on Robustness and Uncertainty of Regularization Methods
• Cut-Thumbnail: A Novel Data Augmentation for Convolutional Neural Network
• MaxUp: A Simple Way to Improve Generalization of Neural Network Training
• ResNet strikes back: An improved training procedure in timm
• Rethinking "Batch" in BatchNorm
• Fixing the train-test resolution discrepancy
• CarveMix: A Simple Data Augmentation Method for Brain Lesion Segmentation
• RAIN: RegulArization on Input and Network for Black-Box Domain Adaptation
• GuidedMixup: An Efficient Mixup Strategy Guided by Saliency Maps
• Semi-Supervised Visual Representation Learning for Fashion Compatibility
• ExtremeC3Net: Extreme Lightweight Portrait Segmentation Networks using Advanced C3-modules
• TDAPNet: Prototype Network with Recurrent Top-Down Attention for Robust Object Classification under Partial Occlusion
• RoIMix: Proposal-Fusion among Multiple Images for Underwater Object Detection
• Artificial and beneficial -- Exploiting artificial images for aerial vehicle detection
• Co-Mixup: Saliency Guided Joint Mixup with Supermodular Diversity
• EnAET: A Self-Trained framework for Semi-Supervised and Supervised Learning with Ensemble Transformations
• Human Attention in Fine-grained Classification
• Interpolation-based semi-supervised learning for object detection
• Removing the Background by Adding the Background: Towards Background Robust Self-supervised Video Representation Learning
• Semantic Segmentation in Multiple Adverse Weather Conditions with Domain Knowledge Retention
• Identifying the Defective: Detecting Damaged Grains for Cereal Appearance Inspection
• Semi-Supervised Domain Generalizable Person Re-Identification
• Cross-directional Feature Fusion Network for Building Damage Assessment from Satellite Imagery
• Making EfficientNet More Efficient: Exploring Batch-Independent Normalization, Group Convolutions and Reduced Resolution Training
• Robust Semantic Segmentation with Superpixel-Mix
• Robust Object Detection under Occlusion with Context-Aware CompositionalNets
• Shape-Texture Debiased Neural Network Training
• Feature Binding with Category-Dependant MixUp for Semantic Segmentation and Adversarial Robustness
• Non-anchor-based vehicle detection for traffic surveillance using bounding ellipses
• F2GAN: Fusing-and-Filling GAN for Few-shot Image Generation
• Counting Out Time: Class Agnostic Video Repetition Counting in the Wild
• Encoding Robustness to Image Style via Adversarial Feature Perturbations
• k-Mixup Regularization for Deep Learning via Optimal Transport
• Spatio-Temporal Attention Mechanism and Knowledge Distillation for Lip Reading
• Regularized Evolutionary Population-Based Training
• Direct Differentiable Augmentation Search
• Compositional Convolutional Neural Networks: A Deep Architecture with Innate Robustness to Partial Occlusion
• The Compact Support Neural Network
• Self-supervised Lesion Change Detection and Localisation in Longitudinal Multiple Sclerosis Brain Imaging
• Robust Instance Segmentation through Reasoning about Multi-Object Occlusion
• DeepSpectrumLite: A Power-Efficient Transfer Learning Framework for Embedded Speech and Audio Processing from Decentralised Data
• Epsilon Consistent Mixup: Structural Regularization with an Adaptive Consistency-Interpolation Tradeoff
• Itsy Bitsy SpiderNet: Fully Connected Residual Network for Fraud Detection
• Improved Robustness of Vision Transformer via PreLayerNorm in Patch Embedding
• ShuffleBlock: Shuffle to Regularize Deep Convolutional Neural Networks
• Scene-aware Learning Network for Radar Object Detection
• Adversarial Data Encryption