FMix: Enhancing Mixed Sample Data Augmentation

Abstract

Mixed Sample Data Augmentation (MSDA) has received increasing attention in recent years, with many successful variants such as MixUp and CutMix. By studying the mutual information between the function learned by a VAE on the original data and on the augmented data we show that MixUp distorts learned functions in a way that CutMix does not. We further demonstrate this by showing that MixUp acts as a form of adversarial training, increasing robustness to attacks such as Deep Fool and Uniform Noise which produce examples similar to those generated by MixUp. We argue that this distortion prevents models from learning about sample specific features in the data, aiding generalisation performance. In contrast, we suggest that CutMix works more like a traditional augmentation, improving performance by preventing memorisation without distorting the data distribution. However, we argue that an MSDA which builds on CutMix to include masks of arbitrary shape, rather than just square, could further prevent memorisation whilst preserving the data distribution in the same way. To this end, we propose FMix, an MSDA that uses random binary masks obtained by applying a threshold to low frequency images sampled from Fourier space. These random masks can take on a wide range of shapes and can be generated for use with one, two, and three dimensional data. FMix improves performance over MixUp and CutMix, without an increase in training time, for a number of models across a range of data sets and problem settings, obtaining a new single model state-of-the-art result on CIFAR-10 without external data. Finally, we show that a consequence of the difference between interpolating MSDA such as MixUp and masking MSDA such as FMix is that the two can be combined to improve performance even further. Code for all experiments is provided at https://github.com/ecs-vlc/FMix .

Code available at https://github.com/ecs-vlc/FMix

References in corpus (23)

• SMOTE: Synthetic Minority Over-sampling Technique
• Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms
• mixup: Beyond Empirical Risk Minimization
• Improved Regularization of Convolutional Neural Networks with Cutout
• Wide Residual Networks
• Random Erasing Data Augmentation
• CutMix: Regularization Strategy to Train Strong Classifiers with Localizable Features
• On Evaluating Adversarial Robustness
• AugMix: A Simple Data Processing Method to Improve Robustness and Uncertainty
• Data Augmentation using Random Image Cropping and Patching for Deep CNNs
• Manifold Mixup: Better Representations by Interpolating Hidden States
• MINE: Mutual Information Neural Estimation
• Data Augmentation by Pairing Samples for Images Classification
• Learning from Between-class Examples for Deep Sound Recognition
• ShakeDrop Regularization for Deep Residual Learning
• Natural Adversarial Examples
• Quantifying the Carbon Emissions of Machine Learning
• SaliencyMix: A Saliency Guided Data Augmentation Strategy for Better Regularization
• Affinity and Diversity: Quantifying Mechanisms of Data Augmentation
• Data Augmentation Revisited: Rethinking the Distribution Gap between Clean and Augmented Data
• How Does Mixup Help With Robustness and Generalization?
• MixUp as Directional Adversarial Training
• On Mixup Regularization

Cited by in corpus (18)

• A Review of Uncertainty Quantification in Deep Learning: Techniques, Applications and Challenges
• SpinalNet: Deep Neural Network with Gradual Input
• Automated data processing and feature engineering for deep learning and big data applications: a survey
• Sharpness-Aware Minimization for Efficiently Improving Generalization
• DSP: Dual Soft-Paste for Unsupervised Domain Adaptive Semantic Segmentation
• i-Mix: A Domain-Agnostic Strategy for Contrastive Representation Learning
• ResizeMix: Mixing Data with Preserved Object Information and True Labels
• Cut-Thumbnail: A Novel Data Augmentation for Convolutional Neural Network
• A Large Multi-Target Dataset of Common Bengali Handwritten Graphemes
• Boosting Discriminative Visual Representation Learning with Scenario-Agnostic Mixup
• ContextMix: A context-aware data augmentation method for industrial visual inspection systems
• Feature Binding with Category-Dependant MixUp for Semantic Segmentation and Adversarial Robustness
• Improving Augmentation and Evaluation Schemes for Semantic Image Synthesis
• Out-distribution aware Self-training in an Open World Setting
• Discover the Mysteries of the Maya: Selected Contributions from the Machine Learning Challenge & The Discovery Challenge Workshop at ECML PKDD 2021
• Adversarial AutoMixup
• Isotonic Data Augmentation for Knowledge Distillation
• Causal Explanations of Image Misclassifications