The worst of both worlds: A comparative analysis of errors in learning from data in psychology and machine learning

Abstract

Recent arguments that machine learning (ML) is facing a reproducibility and replication crisis suggest that some published claims in ML research cannot be taken at face value. These concerns inspire analogies to the replication crisis affecting the social and medical sciences. They also inspire calls for the integration of statistical approaches to causal inference and predictive modeling. A deeper understanding of what reproducibility concerns in supervised ML research have in common with the replication crisis in experimental science puts the new concerns in perspective, and helps researchers avoid "the worst of both worlds," where ML researchers begin borrowing methodologies from explanatory modeling without understanding their limitations and vice versa. We contribute a comparative analysis of concerns about inductive learning that arise in causal attribution as exemplified in psychology versus predictive modeling as exemplified in ML. We identify themes that re-occur in reform discussions, like overreliance on asymptotic theory and non-credible beliefs about real-world data generating processes. We argue that in both fields, claims from learning are implied to generalize outside the specific environment studied (e.g., the input dataset or subject sample, modeling implementation, etc.) but are often impossible to refute due to undisclosed sources of variance in the learning pipeline. In particular, errors being acknowledged in ML expose cracks in long-held beliefs that optimizing predictive accuracy using huge datasets absolves one from having to consider a true data generating process or formally represent uncertainty in performance claims. We conclude by discussing risks that arise when sources of errors are misdiagnosed and the need to acknowledge the role of human inductive biases in learning and reform.

References in corpus (21)

• Explaining and Harnessing Adversarial Examples
• To Explain or to Predict?
• On the Opportunities and Risks of Foundation Models
• The Loss Surfaces of Multilayer Networks
• The prior can generally only be understood in the context of the likelihood
• Identifying and attacking the saddle point problem in high-dimensional non-convex optimization
• Underspecification Presents Challenges for Credibility in Modern Machine Learning
• A Closer Look at Memorization in Deep Networks
• Lessons from Archives: Strategies for Collecting Sociocultural Data in Machine Learning
• Measuring the tendency of CNNs to Learn Surface Statistical Regularities
• Do Datasets Have Politics? Disciplinary Values in Computer Vision Dataset Development
• Fantastic Generalization Measures and Where to Find Them
• In Search of the Real Inductive Bias: On the Role of Implicit Regularization in Deep Learning
• On the Value of Out-of-Distribution Testing: An Example of Goodhart's Law
• A Hierarchy of Limitations in Machine Learning
• SGD on Neural Networks Learns Functions of Increasing Complexity
• Addressing the Loss-Metric Mismatch with Adaptive Loss Alignment
• The piranha problem: Large effects swimming in a small pond
• Targeting Learning: Robust Statistics for Reproducible Research
• Dealing with Disagreements: Looking Beyond the Majority Vote in Subjective Annotations
• Replicability Analysis for Natural Language Processing: Testing Significance with Multiple Datasets

Cited by in corpus (2)

• Ensemble Machine Learning Model Trained on a New Synthesized Dataset Generalizes Well for Stress Prediction Using Wearable Devices
• On "Confirmatory" Methodological Research in Statistics and Related Fields