Improved Regularization of Convolutional Neural Networks with Cutout

Abstract

Convolutional neural networks are capable of learning powerful representational spaces, which are necessary for tackling complex learning tasks. However, due to the model capacity required to capture such representations, they are often susceptible to overfitting and therefore require proper regularization in order to generalize well. In this paper, we show that the simple regularization technique of randomly masking out square regions of input during training, which we call cutout, can be used to improve the robustness and overall performance of convolutional neural networks. Not only is this method extremely easy to implement, but we also demonstrate that it can be used in conjunction with existing forms of data augmentation and other regularizers to further improve model performance. We evaluate this method by applying it to current state-of-the-art architectures on the CIFAR-10, CIFAR-100, and SVHN datasets, yielding new state-of-the-art results of 2.56%, 15.20%, and 1.30% test error respectively. Code is available at https://github.com/uoguelph-mlrg/Cutout

References in corpus (3)

• Improving neural networks by preventing co-adaptation of feature detectors
• Deep Image: Scaling up Image Recognition
• Shake-Shake regularization

Cited by in corpus (26)

• Do ImageNet Classifiers Generalize to ImageNet?
• A Survey on Neural Architecture Search
• Progressive Differentiable Architecture Search: Bridging the Depth Gap between Search and Evaluation
• Further advantages of data augmentation on convolutional neural networks
• Fixup Initialization: Residual Learning Without Normalization
• See Better Before Looking Closer: Weakly Supervised Data Augmentation Network for Fine-Grained Visual Classification
• Improving Robustness Without Sacrificing Accuracy with Patch Gaussian Augmentation
• Data Augmentation for Object Detection via Progressive and Selective Instance-Switching
• Augment your batch: better training with larger batches
• Convolution with even-sized kernels and symmetric padding
• sharpDARTS: Faster and More Accurate Differentiable Architecture Search
• Assume, Augment and Learn: Unsupervised Few-Shot Meta-Learning via Random Labels and Data Augmentation
• Learning Implicitly Recurrent CNNs Through Parameter Sharing
• XNAS: Neural Architecture Search with Expert Advice
• DVOLVER: Efficient Pareto-Optimal Neural Network Architecture Search
• Efficient Forward Architecture Search
• Improving Neural Architecture Search Image Classifiers via Ensemble Learning
• Inductive Transfer for Neural Architecture Optimization
• A Preliminary Study on Data Augmentation of Deep Learning for Image Classification
• Learning Optimal Data Augmentation Policies via Bayesian Optimization for Image Classification Tasks
• Separable Layers Enable Structured Efficient Linear Substitutions
• Genetic Network Architecture Search
• ROI Regularization for Semi-supervised and Supervised Learning
• SwGridNet: A Deep Convolutional Neural Network based on Grid Topology for Image Classification
• Implicit Label Augmentation on Partially Annotated Clips via Temporally-Adaptive Features Learning
• Stochastic Region Pooling: Make Attention More Expressive