Learning Robust Global Representations by Penalizing Local Predictive Power

Abstract

Despite their renowned predictive power on i.i.d. data, convolutional neural networks are known to rely more on high-frequency patterns that humans deem superficial than on low-frequency patterns that agree better with intuitions about what constitutes category membership. This paper proposes a method for training robust convolutional networks by penalizing the predictive power of the local representations learned by earlier layers. Intuitively, our networks are forced to discard predictive signals such as color and texture that can be gleaned from local receptive fields and to rely instead on the global structures of the image. Across a battery of synthetic and benchmark domain adaptation tasks, our method confers improved generalization out of the domain. Also, to evaluate cross-domain transfer, we introduce ImageNet-Sketch, a new dataset consisting of sketch-like images, that matches the ImageNet classification validation set in categories and scale.

References in corpus (12)

• ImageNet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustness
• Domain Separation Networks
• Domain Generalization via Invariant Feature Representation
• Do ImageNet Classifiers Generalize to ImageNet?
• Domain Adaptation for Visual Applications: A Comprehensive Survey
• Measuring the tendency of CNNs to Learn Surface Statistical Regularities
• Co-regularized Alignment for Unsupervised Domain Adaptation
• Learning Robust Representations by Projecting Superficial Statistics Out
• Domain Adaptation with Asymmetrically-Relaxed Distribution Alignment
• Support and Invertibility in Domain-Invariant Representations
• Domain-Invariant Projection Learning for Zero-Shot Recognition
• Multi-Domain Adversarial Learning

Cited by in corpus (38)

• Learning Transferable Visual Models From Natural Language Supervision
• Learning to Prompt for Vision-Language Models
• Reproducible scaling laws for contrastive language-image learning
• EVA-02: A Visual Representation for Neon Genesis
• Natural Adversarial Examples
• The Pitfalls of Simplicity Bias in Neural Networks
• Maximum-Entropy Adversarial Data Augmentation for Improved Generalization and Robustness
• Partial success in closing the gap between human and machine vision
• Frustratingly Simple Domain Generalization via Image Stylization
• Large image datasets: A pyrrhic win for computer vision?
• High Frequency Component Helps Explain the Generalization of Convolutional Neural Networks
• Representation Learning via Invariant Causal Mechanisms
• Unshuffling Data for Improved Generalization
• Assaying Out-Of-Distribution Generalization in Transfer Learning
• Dilated convolution with learnable spacings
• Rethinking Precision of Pseudo Label: Test-Time Adaptation via Complementary Learning
• Expert Training: Task Hardness Aware Meta-Learning for Few-Shot Classification
• Towards Robust Vision Transformer
• Open Domain Generalization with Domain-Augmented Meta-Learning
• Unrestricted Adversarial Attacks on ImageNet Competition
• Shape-Texture Debiased Neural Network Training
• Domain Adaptation Techniques for Natural and Medical Image Classification
• Learning to Diversify for Single Domain Generalization
• Gradual Domain Adaptation in the Wild:When Intermediate Distributions are Absent
• Feature Stylization and Domain-aware Contrastive Learning for Domain Generalization
• Uncertainty-guided Model Generalization to Unseen Domains
• Distance Matters For Improving Performance Estimation Under Covariate Shift
• Cognitive Architecture Toward Common Ground Sharing Among Humans and Generative AIs: Trial on Model-Model Interactions in Tangram Naming Task
• Using Synthetic Corruptions to Measure Robustness to Natural Distribution Shifts
• Self-supervised Benchmark Lottery on ImageNet: Do Marginal Improvements Translate to Improvements on Similar Datasets?
• Why Do Better Loss Functions Lead to Less Transferable Features?
• Untapped Potential of Data Augmentation: A Domain Generalization Viewpoint
• Image Captions are Natural Prompts for Text-to-Image Models
• On-target Adaptation
• Can Targeted Adversarial Examples Transfer When the Source and Target Models Have No Label Space Overlap?
• Reappraising Domain Generalization in Neural Networks
• Discovering Spatial Relationships by Transformers for Domain Generalization
• Counterfactual Supervision-based Information Bottleneck for Out-of-Distribution Generalization