A Hierarchy of Limitations in Machine Learning

Abstract

"All models are wrong, but some are useful", wrote George E. P. Box (1979). Machine learning has focused on the usefulness of probability models for prediction in social systems, but is only now coming to grips with the ways in which these models are wrong---and the consequences of those shortcomings. This paper attempts a comprehensive, structured overview of the specific conceptual, procedural, and statistical limitations of models in machine learning when applied to society. Machine learning modelers themselves can use the described hierarchy to identify possible failure points and think through how to address them, and consumers of machine learning models can know what to question when confronted with the decision about if, where, and how to apply machine learning. The limitations go from commitments inherent in quantification itself, through to showing how unmodeled dependencies can lead to cross-validation being overly optimistic as a way of assessing model performance.

68 pages, 7 figures

References in corpus (15)

• Towards A Rigorous Science of Interpretable Machine Learning
• Understanding Neural Networks Through Deep Visualization
• Interpretable classifiers using rules and Bayesian analysis: Building a better stroke prediction model
• Characterizing the Google Books corpus: Strong limits to inferences of socio-cultural and linguistic evolution
• Deep Learning: A Critical Appraisal
• Model Evaluation, Model Selection, and Algorithm Selection in Machine Learning
• The Hidden Assumptions Behind Counterfactual Explanations and Principal Reasons
• What do you learn from context? Probing for sentence structure in contextualized word representations
• Big Questions for Social Media Big Data: Representativeness, Validity and Other Methodological Pitfalls
• Machine Learning that Matters
• Statistical Inference: The Big Picture
• What's Sex Got To Do With Fair Machine Learning?
• On Individual Risk
• Cross-Validation, Risk Estimation, and Model Selection
• Classifier comparison using precision

Cited by in corpus (5)

• Rethinking Fairness: An Interdisciplinary Survey of Critiques of Hegemonic ML Fairness Approaches
• How to avoid machine learning pitfalls: a guide for academic researchers
• Limitations of machine learning for building energy prediction: ASHRAE Great Energy Predictor III Kaggle competition error analysis
• The worst of both worlds: A comparative analysis of errors in learning from data in psychology and machine learning
• Measuring justice in machine learning