A Bayesian Data Augmentation Approach for Learning Deep Models

Abstract

Data augmentation is an essential part of the training process applied to deep learning models. The motivation is that a robust training process for deep learning models depends on large annotated datasets, which are expensive to be acquired, stored and processed. Therefore a reasonable alternative is to be able to automatically generate new annotated training samples using a process known as data augmentation. The dominant data augmentation approach in the field assumes that new training samples can be obtained via random geometric or appearance transformations applied to annotated training samples, but this is a strong assumption because it is unclear if this is a reliable generative model for producing new training samples. In this paper, we provide a novel Bayesian formulation to data augmentation, where new annotated training points are treated as missing variables and generated based on the distribution learned from the training set. For learning, we introduce a theoretically sound algorithm --- generalised Monte Carlo expectation maximisation, and demonstrate one possible implementation via an extension of the Generative Adversarial Network (GAN). Classification results on MNIST, CIFAR-10 and CIFAR-100 show the better performance of our proposed method compared to the current dominant data augmentation approach mentioned above --- the results also show that our approach produces better classification results than similar GAN models.

Accepted to NISP 2017

References in corpus (6)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Conditional Generative Adversarial Nets
• Conditional Image Synthesis With Auxiliary Classifier GANs
• InfoGAN: Interpretable Representation Learning by Information Maximizing Generative Adversarial Nets
• Text Understanding from Scratch
• Triple Generative Adversarial Nets

Cited by in corpus (30)

• Albumentations: fast and flexible image augmentations
• AutoAugment: Learning Augmentation Policies from Data
• A Comprehensive Survey of Image Augmentation Techniques for Deep Learning
• Cross-Domain Few-Shot Classification via Learned Feature-Wise Transformation
• Recent Advances in Object Detection in the Age of Deep Convolutional Neural Networks
• Active learning for data streams: a survey
• Affinity and Diversity: Quantifying Mechanisms of Data Augmentation
• Data Augmentation Revisited: Rethinking the Distribution Gap between Clean and Augmented Data
• Multi-modal Cycle-consistent Generalized Zero-Shot Learning
• Optical Gaze Tracking with Spatially-Sparse Single-Pixel Detectors
• Viewmaker Networks: Learning Views for Unsupervised Representation Learning
• Neural Transformation Learning for Deep Anomaly Detection Beyond Images
• FBNetV3: Joint Architecture-Recipe Search using Predictor Pretraining
• Domain Randomization and Pyramid Consistency: Simulation-to-Real Generalization without Accessing Target Domain Data
• Simultaneous synthesis of FLAIR and segmentation of white matter hypointensities from T1 MRIs
• PointAugment: an Auto-Augmentation Framework for Point Cloud Classification
• SCARF: Self-Supervised Contrastive Learning using Random Feature Corruption
• Online Hyper-parameter Learning for Auto-Augmentation Strategy
• Adversarial Learning of General Transformations for Data Augmentation
• DADA: Deep Adversarial Data Augmentation for Extremely Low Data Regime Classification
• Generative Models For Deep Learning with Very Scarce Data
• McKernel: A Library for Approximate Kernel Expansions in Log-linear Time
• An Improved Data Augmentation Scheme for Model Predictive Control Policy Approximation
• A Sensitivity-based Data Augmentation Framework for Model Predictive Control Policy Approximation
• How good is my GAN?
• De-Pois: An Attack-Agnostic Defense against Data Poisoning Attacks
• Local Patch AutoAugment with Multi-Agent Collaboration
• What augmentations are sensitive to hyper-parameters and why?
• Rapid Risk Minimization with Bayesian Models Through Deep Learning Approximation
• Safe Augmentation: Learning Task-Specific Transformations from Data