Shortcut Learning in Deep Neural Networks

Abstract

Deep learning has triggered the current rise of artificial intelligence and is the workhorse of today's machine intelligence. Numerous success stories have rapidly spread all over science, industry and society, but its limitations have only recently come into focus. In this perspective we seek to distill how many of deep learning's problems can be seen as different symptoms of the same underlying problem: shortcut learning. Shortcuts are decision rules that perform well on standard benchmarks but fail to transfer to more challenging testing conditions, such as real-world scenarios. Related issues are known in Comparative Psychology, Education and Linguistics, suggesting that shortcut learning may be a common characteristic of learning systems, biological and artificial alike. Based on these observations, we develop a set of recommendations for model interpretation and benchmarking, highlighting recent advances in machine learning to improve robustness and transferability from the lab to real-world applications.

perspective article published at Nature Machine Intelligence (https://doi.org/10.1038/s42256-020-00257-z)

References in corpus (26)

• Equality of Opportunity in Supervised Learning
• CheXNet: Radiologist-Level Pneumonia Detection on Chest X-Rays with Deep Learning
• Unmasking Clever Hans Predictors and Assessing What Machines Really Learn
• DeepStack: Expert-Level Artificial Intelligence in No-Limit Poker
• Solving Rubik's Cube with a Robot Hand
• Do ImageNet Classifiers Generalize to ImageNet?
• Deep Learning: A Critical Appraisal
• A Closer Look at Memorization in Deep Networks
• Benchmarking Neural Network Robustness to Common Corruptions and Perturbations
• CARLA: An Open Urban Driving Simulator
• Measuring the tendency of CNNs to Learn Surface Statistical Regularities
• Approximating CNNs with Bag-of-local-Features models works surprisingly well on ImageNet
• Flexibly Fair Representation Learning by Disentanglement
• On Causal and Anticausal Learning
• Men Also Like Shopping: Reducing Gender Bias Amplification using Corpus-level Constraints
• Towards Understanding Generalization of Deep Learning: Perspective of Loss Landscapes
• A Meta-Transfer Objective for Learning to Disentangle Causal Mechanisms
• Right for the Wrong Reasons: Diagnosing Syntactic Heuristics in Natural Language Inference
• The Pitfalls of Simplicity Bias in Neural Networks
• What shapes feature representations? Exploring datasets, architectures, and training
• SGD on Neural Networks Learns Functions of Increasing Complexity
• Automatic Shortcut Removal for Self-Supervised Representation Learning
• Visual Concepts and Compositional Voting
• On the Measure of Intelligence
• Generating captions without looking beyond objects
• When Choosing Plausible Alternatives, Clever Hans can be Clever

Cited by in corpus (237)

• Learning Transferable Visual Models From Natural Language Supervision
• On the Opportunities and Risks of Foundation Models
• Domain Generalization: A Survey
• Explainable Artificial Intelligence (XAI) 2.0: A Manifesto of Open Challenges and Interdisciplinary Research Directions
• Metrics reloaded: Recommendations for image analysis validation
• Underspecification Presents Challenges for Credibility in Modern Machine Learning
• The Debate Over Understanding in AI's Large Language Models
• Will Artificial Intelligence supersede Earth System and Climate Models?
• Measuring Massive Multitask Language Understanding
• WILDS: A Benchmark of in-the-Wild Distribution Shifts
• A Primer on Motion Capture with Deep Learning: Principles, Pitfalls and Perspectives
• Causal Attention for Interpretable and Generalizable Graph Classification
• Ground Truth Evaluation of Neural Network Explanations with CLEVR-XAI
• Bridging observation, theory and numerical simulation of the ocean using Machine Learning
• Improving robustness against common corruptions by covariate shift adaptation
• Large Language Models as Zero-Shot Conversational Recommenders
• Deconfounded Image Captioning: A Causal Retrospect
• Abstraction and Analogy-Making in Artificial Intelligence
• Exploring the Limits of Out-of-Distribution Detection
• Materials science in the era of large language models: a perspective
• Detecting Shortcut Learning for Fair Medical AI using Shortcut Testing
• Synthetic Datasets for Autonomous Driving: A Survey
• ADAM Challenge: Detecting Age-related Macular Degeneration from Fundus Images
• Speciesist bias in AI -- How AI applications perpetuate discrimination and unfair outcomes against animals
• CLIP in Medical Imaging: A Survey
• Recommendations on test datasets for evaluating AI solutions in pathology
• On the Value of Out-of-Distribution Testing: An Example of Goodhart's Law
• Mapping global dynamics of benchmark creation and saturation in artificial intelligence
• Responsible and Regulatory Conform Machine Learning for Medicine: A Survey of Challenges and Solutions
• Black-Box Access is Insufficient for Rigorous AI Audits
• Human Perception of Audio Deepfakes
• Partial success in closing the gap between human and machine vision
• Counterfactual Reasoning for Out-of-distribution Multimodal Sentiment Analysis
• Transformers in Self-Supervised Monocular Depth Estimation with Unknown Camera Intrinsics
• Self-Supervised Learning with Data Augmentations Provably Isolates Content from Style
• Data and its (dis)contents: A survey of dataset development and use in machine learning research
• Improving deep neural network generalization and robustness to background bias via layer-wise relevance propagation optimization
• Understanding the Failure Modes of Out-of-Distribution Generalization
• What shapes feature representations? Exploring datasets, architectures, and training
• Learning Debiased Representation via Disentangled Feature Augmentation
• Image Classification with Small Datasets: Overview and Benchmark
• AutoPrognosis 2.0: Democratizing Diagnostic and Prognostic Modeling in Healthcare with Automated Machine Learning
• Invariance Principle Meets Information Bottleneck for Out-of-Distribution Generalization
• Measuring the Accuracy of Automatic Speech Recognition Solutions
• Gradient Starvation: A Learning Proclivity in Neural Networks
• What do Models Learn from Question Answering Datasets?
• Network Intrusion Datasets: A Survey, Limitations, and Recommendations
• On Disentangled Representations Learned From Correlated Data
• Gender and Racial Bias in Visual Question Answering Datasets
• The worst of both worlds: A comparative analysis of errors in learning from data in psychology and machine learning
• From Learning to Relearning: A Framework for Diminishing Bias in Social Robot Navigation
• Mind the gap: Challenges of deep learning approaches to Theory of Mind
• The Clever Hans Effect in Unsupervised Learning
• Large Language Models Can be Lazy Learners: Analyze Shortcuts in In-Context Learning
• When is invariance useful in an Out-of-Distribution Generalization problem ?
• CLOOB: Modern Hopfield Networks with InfoLOOB Outperform CLIP
• Learning Disentangled Behaviour Patterns for Wearable-based Human Activity Recognition
• Counterfactual Generative Networks
• Discrete and continuous representations and processing in deep learning: Looking forward
• Fairness via Representation Neutralization
• Cross-Domain Object Detection Using Unsupervised Image Translation
• Exposing Previously Undetectable Faults in Deep Neural Networks
• Beyond accuracy: quantifying trial-by-trial behaviour of CNNs and humans by measuring error consistency
• Emergent Properties of Foveated Perceptual Systems
• Factual Probing Is [MASK]: Learning vs. Learning to Recall
• Poisoning the Unlabeled Dataset of Semi-Supervised Learning
• Multi-channel learning for integrating structural hierarchies into context-dependent molecular representation
• Learning Robust Representation for Joint Grading of Ophthalmic Diseases via Adaptive Curriculum and Feature Disentanglement
• On the surprising similarities between supervised and self-supervised models
• Reduced, Reused and Recycled: The Life of a Dataset in Machine Learning Research
• Neuromorphic Visual Scene Understanding with Resonator Networks
• Perceptron Theory Can Predict the Accuracy of Neural Networks
• Why we need biased AI -- How including cognitive and ethical machine biases can enhance AI systems
• The Sweet Danger of Sugar: Debunking Representation Learning for Encrypted Traffic Classification
• Analyzing Atomic Interactions in Molecules as Learned by Neural Networks
• Towards Human-like Perception: Learning Structural Causal Model in Heterogeneous Graph
• Object-aware Contrastive Learning for Debiased Scene Representation
• Can Subnetwork Structure be the Key to Out-of-Distribution Generalization?
• Out-of-distribution Prediction with Invariant Risk Minimization: The Limitation and An Effective Fix
• Causal Inference in Natural Language Processing: Estimation, Prediction, Interpretation and Beyond
• Assessing out-of-domain generalization for robust building damage detection
• Fairness through Aleatoric Uncertainty
• Multi-Scale and Multi-Layer Contrastive Learning for Domain Generalization
• Kriging prior Regression: A Case for Kriging-Based Spatial Features with TabPFN in Soil Mapping
• Linear unit-tests for invariance discovery
• Understanding the computational demands underlying visual reasoning
• Time for a Background Check! Uncovering the impact of Background Features on Deep Neural Networks
• Physion: Evaluating Physical Prediction from Vision in Humans and Machines
• ShortcutLens: A Visual Analytics Approach for Exploring Shortcuts in Natural Language Understanding Dataset
• Spectral decoupling allows training transferable neural networks in medical imaging
• Building Human-like Communicative Intelligence: A Grounded Perspective
• Towards Interpreting and Mitigating Shortcut Learning Behavior of NLU Models
• Achieve Fairness without Demographics for Dermatological Disease Diagnosis
• Causality and Independence Enhancement for Biased Node Classification
• How benign is benign overfitting?
• Identifying a Training-Set Attack's Target Using Renormalized Influence Estimation
• Deep Learning Methods for Abstract Visual Reasoning: A Survey on Raven's Progressive Matrices
• Simple data balancing achieves competitive worst-group-accuracy
• Which Shortcut Cues Will DNNs Choose? A Study from the Parameter-Space Perspective
• Environment Inference for Invariant Learning
• In Search of netUnicorn: A Data-Collection Platform to Develop Generalizable ML Models for Network Security Problems
• Eight challenges in developing theory of intelligence
• False Sense of Security: Leveraging XAI to Analyze the Reasoning and True Performance of Context-less DGA Classifiers
• Classification-based detection and quantification of cross-domain data bias in materials discovery
• Latent Adversarial Debiasing: Mitigating Collider Bias in Deep Neural Networks
• Overinterpretation reveals image classification model pathologies
• Generalized Semantic Contrastive Learning via Embedding Side Information for Few-Shot Object Detection
• How You Split Matters: Data Leakage and Subject Characteristics Studies in Longitudinal Brain MRI Analysis
• Benign Shortcut for Debiasing: Fair Visual Recognition via Intervention with Shortcut Features
• Size-Invariant Graph Representations for Graph Classification Extrapolations
• TextFlint: Unified Multilingual Robustness Evaluation Toolkit for Natural Language Processing
• Guiding Visual Attention in Deep Convolutional Neural Networks Based on Human Eye Movements
• The benefits and costs of explainable artificial intelligence in visual quality control: Evidence from fault detection performance and eye movements
• PointMask: Towards Interpretable and Bias-Resilient Point Cloud Processing
• Adversarial NLI for Factual Correctness in Text Summarisation Models
• Fairness and Robustness in Invariant Learning: A Case Study in Toxicity Classification
• PreferenceNet: Encoding Human Preferences in Auction Design with Deep Learning
• Uncovering Tidal Treasures: Automated Classification of Faint Tidal Features in DECaLS Data
• Do humans and Convolutional Neural Networks attend to similar areas during scene classification: Effects of task and image type
• Quantitatively Measuring and Contrastively Exploring Heterogeneity for Domain Generalization
• Causally motivated Shortcut Removal Using Auxiliary Labels
• Study on the Helpfulness of Explainable Artificial Intelligence
• Exploring Inconsistent Knowledge Distillation for Object Detection with Data Augmentation
• Robustness testing of AI systems: A case study for traffic sign recognition
• How to Train your Antivirus: RL-based Hardening through the Problem-Space
• Preemptively Pruning Clever-Hans Strategies in Deep Neural Networks
• A Generalizable Deep Learning System for Cardiac MRI
• Visual Representation Learning Does Not Generalize Strongly Within the Same Domain
• Implicit Regularization via Neural Feature Alignment
• Reduced Implication-bias Logic Loss for Neuro-Symbolic Learning
• Usefulness of interpretability methods to explain deep learning based plant stress phenotyping
• Robust Semantic Segmentation with Superpixel-Mix
• Multimodal Multi-User Surface Recognition with the Kernel Two-Sample Test
• Probabilistic Numeric Convolutional Neural Networks
• An Invitation to Deep Reinforcement Learning
• A Scaling Law for Synthetic-to-Real Transfer: How Much Is Your Pre-training Effective?
• Environment Invariant Linear Least Squares
• Learning Debiased and Disentangled Representations for Semantic Segmentation
• Supervising the Transfer of Reasoning Patterns in VQA
• Shape-Texture Debiased Neural Network Training
• Towards Audit Requirements for AI-based Systems in Mobility Applications
• Towards Principled Disentanglement for Domain Generalization
• Robustness Challenges in Model Distillation and Pruning for Natural Language Understanding
• Mitigating Modality Bias in Multi-modal Entity Alignment from a Causal Perspective
• Disentangling representations of retinal images with generative models
• Hierarchically Compositional Tasks and Deep Convolutional Networks
• RobustPointSet: A Dataset for Benchmarking Robustness of Point Cloud Classifiers
• BiaSwap: Removing dataset bias with bias-tailored swapping augmentation
• Challenges for cognitive decoding using deep learning methods
• Deep Neural Models for color discrimination and color constancy
• Data augmentation and image understanding
• Overcoming Statistical Shortcuts for Open-ended Visual Counting
• Assessment of the Reliablity of a Model's Decision by Generalizing Attribution to the Wavelet Domain
• Mask of truth: model sensitivity to unexpected regions of medical images
• Fairness Mediator: Neutralize Stereotype Associations to Mitigate Bias in Large Language Models
• Improving the Robustness of QA Models to Challenge Sets with Variational Question-Answer Pair Generation
• Clarify: Improving Model Robustness With Natural Language Corrections
• HCVP: Leveraging Hierarchical Contrastive Visual Prompt for Domain Generalization
• Identifying Critical Tokens for Accurate Predictions in Transformer-based Medical Imaging Models
• The Impact of Feature Representation on the Accuracy of Photonic Neural Networks
• BERT is to NLP what AlexNet is to CV: Can Pre-Trained Language Models Identify Analogies?
• Global Wheat Challenge 2020: Analysis of the competition design and winning models
• Tracking Without Re-recognition in Humans and Machines
• Causality-inspired Single-source Domain Generalization for Medical Image Segmentation
• Source-Free Adaptation to Measurement Shift via Bottom-Up Feature Restoration
• Explicitly Representing Syntax Improves Sentence-to-layout Prediction of Unexpected Situations
• Dataset Distribution Impacts Model Fairness: Single vs. Multi-Task Learning
• Simplicity Bias Leads to Amplified Performance Disparities
• Transformers in Unsupervised Structure-from-Motion
• Approaching human 3D shape perception with neurally mappable models
• Clean-image Backdoor Attacks
• ViG-Bias: Visually Grounded Bias Discovery and Mitigation
• Exploring connections of spectral analysis and transfer learning in medical imaging
• Communication Access Real-Time Translation Through Collaborative Correction of Automatic Speech Recognition
• Going Grayscale: The Road to Understanding and Improving Unlearnable Examples
• Robust Representation Learning via Perceptual Similarity Metrics
• Tasting the cake: evaluating self-supervised generalization on out-of-distribution multimodal MRI data
• Rotation-Invariant Autoencoders for Signals on Spheres
• Mitigating Gender Bias in Captioning Systems
• Selective Forgetting of Deep Networks at a Finer Level than Samples
• TIMEDIAL: Temporal Commonsense Reasoning in Dialog
• Multimodal Scale Consistency and Awareness for Monocular Self-Supervised Depth Estimation
• Layer Pruning with Consensus: A Triple-Win Solution
• Towards Adversarially Robust and Domain Generalizable Stereo Matching by Rethinking DNN Feature Backbones
• Integrating Intrinsic and Extrinsic Explainability: The Relevance of Understanding Neural Networks for Human-Robot Interaction
• Concept Extraction for Time Series with ECLAD-ts
• Geometry matters: Exploring language examples at the decision boundary
• Fighting Copycat Agents in Behavioral Cloning from Observation Histories
• From Physics to Representation: Audio Learning with Synthetic Pre-training via Procedural Generation
• Interpreting A Pre-trained Model Is A Key For Model Architecture Optimization: A Case Study On Wav2Vec 2.0
• Temporally Resolution Decrement: Utilizing the Shape Consistency for Higher Computational Efficiency
• ImageNet Pre-training also Transfers Non-Robustness
• Towards Robust Classification Model by Counterfactual and Invariant Data Generation
• Fermi-Bose Machine achieves both generalization and adversarial robustness
• Conditional Adversarial Camera Model Anonymization
• None of the Others: a General Technique to Distinguish Reasoning from Memorization in Multiple-Choice LLM Evaluation Benchmarks
• Causal thinking for decision making on Electronic Health Records: why and how
• Echoes: Unsupervised Debiasing via Pseudo-bias Labeling in an Echo Chamber
• Faster ISNet for Background Bias Mitigation on Deep Neural Networks
• The distance between the weights of the neural network is meaningful
• Detecting Spurious Correlations with Sanity Tests for Artificial Intelligence Guided Radiology Systems
• Understanding the Logit Distributions of Adversarially-Trained Deep Neural Networks
• Towards Desiderata-Driven Design of Visual Counterfactual Explainers
• FOCUS: Familiar Objects in Common and Uncommon Settings
• Machine Learning Featurizations for AI Hacking of Political Systems
• Neural Networks for Learning Counterfactual G-Invariances from Single Environments
• Unsupervised Learning of Debiased Representations with Pseudo-Attributes
• Pose Discrepancy Spatial Transformer Based Feature Disentangling for Partial Aspect Angles SAR Target Recognition
• Combining Diverse Feature Priors
• HOLMES: HOLonym-MEronym based Semantic inspection for Convolutional Image Classifiers
• Requirement analysis for an artificial intelligence model for the diagnosis of the COVID-19 from chest X-ray data
• Learning Modular Structures That Generalize Out-of-Distribution
• Towards Unbiased Visual Emotion Recognition via Causal Intervention
• Foundation models on the bridge: Semantic hazard detection and safety maneuvers for maritime autonomy with vision-language models
• Improving Robustness to Out-of-Distribution States in Imitation Learning via Deep Koopman-Boosted Diffusion Policy
• White Paper Assistance: A Step Forward Beyond the Shortcut Learning
• Self-supervision of Feature Transformation for Further Improving Supervised Learning
• Efficient Reasoning Distillation: Small Video-Language Models via Synthetic CoT and Difficulty-Aware Fine-Tuning
• Improving statistical precision in Monte Carlo samples with negative weights via reweighting and uncertainty quantification
• Achieving Domain Robustness in Stereo Matching Networks by Removing Shortcut Learning
• A Trustworthy By Design Classification Model for Building Energy Retrofit Decision Support
• Toward Building Science Discovery Machines
• Revisiting Out-of-Distribution Detection in Real-time Object Detection: From Benchmark Pitfalls to a New Mitigation Paradigm
• Identifying and Exploiting Structures for Reliable Deep Learning
• Birds look like cars: Adversarial analysis of intrinsically interpretable deep learning
• Language Modeling, Lexical Translation, Reordering: The Training Process of NMT through the Lens of Classical SMT
• Debiasing Methods in Natural Language Understanding Make Bias More Accessible
• A Closer Look at Few-Shot Crosslingual Transfer: The Choice of Shots Matters
• Can Question Generation Debias Question Answering Models? A Case Study on Question-Context Lexical Overlap
• The Pragmatic Frames of Spurious Correlations in Machine Learning: Interpreting How and Why They Matter
• Counterfactual Supervision-based Information Bottleneck for Out-of-Distribution Generalization
• Learning Less Generalizable Patterns with an Asymmetrically Trained Double Classifier for Better Test-Time Adaptation
• Ridge Rider: Finding Diverse Solutions by Following Eigenvectors of the Hessian
• Adapting Machine Learning Diagnostic Models to New Populations Using a Small Amount of Data: Results from Clinical Neuroscience
• Shape Defense Against Adversarial Attacks
• Human-Level Accuracy, Non-Human Strategies: Revealing Model-Human Divergence in Video Physical Reasoning
• A Hormetic Approach to the Value-Loading Problem: Preventing the Paperclip Apocalypse?