Improving robustness against common corruptions by covariate shift adaptation

Abstract

Today's state-of-the-art machine vision models are vulnerable to image corruptions like blurring or compression artefacts, limiting their performance in many real-world applications. We here argue that popular benchmarks to measure model robustness against common corruptions (like ImageNet-C) underestimate model robustness in many (but not all) application scenarios. The key insight is that in many scenarios, multiple unlabeled examples of the corruptions are available and can be used for unsupervised online adaptation. Replacing the activation statistics estimated by batch normalization on the training set with the statistics of the corrupted images consistently improves the robustness across 25 different popular computer vision models. Using the corrected statistics, ResNet-50 reaches 62.2% mCE on ImageNet-C compared to 76.7% without adaptation. With the more robust DeepAugment+AugMix model, we improve the state of the art achieved by a ResNet50 model up to date from 53.6% mCE to 45.4% mCE. Even adapting to a single sample improves robustness for the ResNet-50 and AugMix models, and 32 samples are sufficient to improve the current state of the art for a ResNet-50 architecture. We argue that results with adapted statistics should be included whenever reporting scores in corruption benchmarks and other out-of-distribution generalization settings.

Accepted at the Thirty-fourth Conference on Neural Information Processing Systems. Web: https://domainadaptation.org/batchnorm/

References in corpus (20)

• Shortcut Learning in Deep Neural Networks
• AutoAugment: Learning Augmentation Policies from Data
• ImageNet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustness
• Learning multiple visual domains with residual adapters
• Big Self-Supervised Models are Strong Semi-Supervised Learners
• Generalisation in humans and deep neural networks
• Do ImageNet Classifiers Generalize to ImageNet?
• Revisiting Batch Normalization For Practical Domain Adaptation
• Making Convolutional Networks Shift-Invariant Again
• Natural Adversarial Examples
• Adversarial Examples Are a Natural Consequence of Test Error in Noise
• Benchmarking Robustness in Object Detection: Autonomous Driving when Winter is Coming
• On Causal and Anticausal Learning
• Fixup Initialization: Residual Learning Without Normalization
• Evaluating Prediction-Time Batch Normalization for Robustness under Covariate Shift
• Training BatchNorm and Only BatchNorm: On the Expressive Power of Random Features in CNNs
• Batch Normalization is a Cause of Adversarial Vulnerability
• Compounding the Performance Improvements of Assembled Techniques in a Convolutional Neural Network
• Robustness properties of Facebook's ResNeXt WSL models
• Identifying Statistical Bias in Dataset Replication

Cited by in corpus (22)

• A Comprehensive Survey on Test-Time Adaptation under Distribution Shifts
• MEMO: Test Time Robustness via Adaptation and Augmentation
• GALA: Graph Diffusion-based Alignment with Jigsaw for Source-free Domain Adaptation
• SALUDA: Surface-based Automotive Lidar Unsupervised Domain Adaptation
• Robust Image Classification Using A Low-Pass Activation Function and DCT Augmentation
• DARTSRepair: Core-failure-set Guided DARTS for Network Robustness to Common Corruptions
• Adaptive Denoising via GainTuning
• AdaShadow: Responsive Test-time Model Adaptation in Non-stationary Mobile Environments
• Adaptive Domain Generalization via Online Disagreement Minimization
• A Bayesian Approach to OOD Robustness in Image Classification
• Test-Time Adaptation for Nighttime Color-Thermal Semantic Segmentation
• Behavior of Keyword Spotting Networks Under Noisy Conditions
• Addressing Data Distribution Shifts in Online Machine Learning Powered Smart City Applications Using Augmented Test-Time Adaptation
• Covariance-aware Feature Alignment with Pre-computed Source Statistics for Test-time Adaptation to Multiple Image Corruptions
• Post-Training BatchNorm Recalibration
• DIRA: Dynamic Domain Incremental Regularised Adaptation
• Uncover and Unlearn Nuisances: Agnostic Fully Test-Time Adaptation
• Self-supervised Benchmark Lottery on ImageNet: Do Marginal Improvements Translate to Improvements on Similar Datasets?
• On-target Adaptation
• Test-Time Adaptation with Principal Component Analysis
• Learning Less Generalizable Patterns with an Asymmetrically Trained Double Classifier for Better Test-Time Adaptation
• GIPSO: Geometrically Informed Propagation for Online Adaptation in 3D LiDAR Segmentation