Learning explanations that are hard to vary

Abstract

In this paper, we investigate the principle that `good explanations are hard to vary' in the context of deep learning. We show that averaging gradients across examples -- akin to a logical OR of patterns -- can favor memorization and `patchwork' solutions that sew together different strategies, instead of identifying invariances. To inspect this, we first formalize a notion of consistency for minima of the loss surface, which measures to what extent a minimum appears only when examples are pooled. We then propose and experimentally validate a simple alternative algorithm based on a logical AND, that focuses on invariances and prevents memorization in a set of real-world tasks. Finally, using a synthetic dataset with a clear distinction between invariant and spurious mechanisms, we dissect learning signals and compare this approach to well-established regularizers.

From v1: extended 2.2 and 2.3, added details for reproducibility and link to codebase

References in corpus (11)

• Domain Generalization via Invariant Feature Representation
• Invariant Risk Minimization
• Three Factors Influencing Minima in SGD
• Leveraging Procedural Generation to Benchmark Reinforcement Learning
• Invariant Models for Causal Transfer Learning
• Gradient Descent Converges to Minimizers
• Adapting Auxiliary Losses Using Gradient Similarity
• Learning Independent Causal Mechanisms
• Stiffness: A New Perspective on Generalization in Neural Networks
• Learning Novel Policies For Tasks
• Gradient Agreement as an Optimization Objective for Meta-Learning

Cited by in corpus (21)

• Active label cleaning for improved dataset quality under resource constraints
• What shapes feature representations? Exploring datasets, architectures, and training
• Gradient Starvation: A Learning Proclivity in Neural Networks
• Towards a Theoretical Framework of Out-of-Distribution Generalization
• When is invariance useful in an Out-of-Distribution Generalization problem ?
• Assaying Out-Of-Distribution Generalization in Transfer Learning
• SAND-mask: An Enhanced Gradient Masking Strategy for the Discovery of Invariances in Domain Generalization
• Can Subnetwork Structure be the Key to Out-of-Distribution Generalization?
• Saliency is a Possible Red Herring When Diagnosing Poor Generalization
• Linear unit-tests for invariance discovery
• Local Learning Matters: Rethinking Data Heterogeneity in Federated Learning
• A Theory of Label Propagation for Subpopulation Shift
• Accounting for Unobserved Confounding in Domain Generalization
• Using Positive Matching Contrastive Loss with Facial Action Units to mitigate bias in Facial Expression Recognition
• Learning Representations that Support Robust Transfer of Predictors
• Model-Invariant State Abstractions for Model-Based Reinforcement Learning
• An Information-theoretic Approach to Distribution Shifts
• On Invariance Penalties for Risk Minimization
• Beyond Discriminant Patterns: On the Robustness of Decision Rule Ensembles
• Gradient Masked Federated Optimization
• Matched sample selection with GANs for mitigating attribute confounding