On Interaction Between Augmentations and Corruptions in Natural Corruption Robustness

Abstract

Invariance to a broad array of image corruptions, such as warping, noise, or color shifts, is an important aspect of building robust models in computer vision. Recently, several new data augmentations have been proposed that significantly improve performance on ImageNet-C, a benchmark of such corruptions. However, there is still a lack of basic understanding on the relationship between data augmentations and test-time corruptions. To this end, we develop a feature space for image transforms, and then use a new measure in this space between augmentations and corruptions called the Minimal Sample Distance to demonstrate a strong correlation between similarity and performance. We then investigate recent data augmentations and observe a significant degradation in corruption robustness when the test-time corruptions are sampled to be perceptually dissimilar from ImageNet-C in this feature space. Our results suggest that test error can be improved by training on perceptually similar augmentations, and data augmentations may not generalize well beyond the existing benchmark. We hope our results and tools will allow for more robust progress towards improving robustness to image corruptions. We provide code at https://github.com/facebookresearch/augmentation-corruption.

arXiv commentCode available at https://github.com/facebookresearch/augmentation-corruption

References in corpus (13)

• Accurate, Large Minibatch SGD: Training ImageNet in 1 Hour
• Image Quality Assessment: Unifying Structure and Texture Similarity
• The Unreasonable Effectiveness of Deep Features as a Perceptual Metric
• ImageNet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustness
• Generalisation in humans and deep neural networks
• Do ImageNet Classifiers Generalize to ImageNet?
• Tent: Fully Test-time Adaptation by Entropy Minimization
• Do CIFAR-10 Classifiers Generalize to CIFAR-10?
• A Kernel Theory of Modern Data Augmentation
• Improving Robustness Without Sacrificing Accuracy with Patch Gaussian Augmentation
• Compounding the Performance Improvements of Assembled Techniques in a Convolutional Neural Network
• A Study and Comparison of Human and Deep Learning Recognition Performance Under Visual Distortions
• Robustness properties of Facebook's ResNeXt WSL models

Cited by in corpus (3)

• Test-Time Adaptation to Distribution Shift by Confidence Maximization and Input Transformation
• Certified Adversarial Defenses Meet Out-of-Distribution Corruptions: Benchmarking Robustness and Simple Baselines
• A Modular System for Enhanced Robustness of Multimedia Understanding Networks via Deep Parametric Estimation