Further advantages of data augmentation on convolutional neural networks

Abstract

Data augmentation is a popular technique largely used to enhance the training of convolutional neural networks. Although many of its benefits are well known by deep learning researchers and practitioners, its implicit regularization effects, as compared to popular explicit regularization techniques, such as weight decay and dropout, remain largely unstudied. As a matter of fact, convolutional neural networks for image object classification are typically trained with both data augmentation and explicit regularization, assuming the benefits of all techniques are complementary. In this paper, we systematically analyze these techniques through ablation studies of different network architectures trained with different amounts of training data. Our results unveil a largely ignored advantage of data augmentation: networks trained with just data augmentation more easily adapt to different architectures and amount of training data, as opposed to weight decay and dropout, which require specific fine-tuning of their hyperparameters.

arXiv commentPreprint of the manuscript accepted for presentation at the International Conference on Artificial Neural Networks (ICANN) 2018. Best Paper Award

References in corpus (12)

• Improved Regularization of Convolutional Neural Networks with Cutout
• Striving for Simplicity: The All Convolutional Net
• The Effectiveness of Data Augmentation in Image Classification using Deep Learning
• Deep Big Simple Neural Nets Excel on Handwritten Digit Recognition
• Data Augmentation Generative Adversarial Networks
• Smart Augmentation - Learning an Optimal Data Augmentation Strategy
• Fractional Max-Pooling
• Deep Image: Scaling up Image Recognition
• Learning to Compose Domain-Specific Transformations for Data Augmentation
• Few-Shot Learning with Metric-Agnostic Conditional Embeddings
• Dataset Augmentation in Feature Space
• Do deep nets really need weight decay and dropout?

Cited by in corpus (10)

• Albumentations: fast and flexible image augmentations
• Affinity and Diversity: Quantifying Mechanisms of Data Augmentation
• Randomness In Neural Network Training: Characterizing The Impact of Tooling
• EllSeg-Gen, towards Domain Generalization for head-mounted eyetracking
• Assessing Intelligence in Artificial Neural Networks
• Domain Generalization via Inference-time Label-Preserving Target Projections
• Discrepancy Minimization in Domain Generalization with Generative Nearest Neighbors
• A Probabilistic Framework for Knowledge Graph Data Augmentation
• Robustness and Overfitting Behavior of Implicit Background Models
• Generalized Categorisation of Digital Pathology Whole Image Slides using Unsupervised Learning