A Framework for Understanding Sources of Harm throughout the Machine Learning Life Cycle

Abstract

As machine learning (ML) increasingly affects people and society, awareness of its potential unwanted consequences has also grown. To anticipate, prevent, and mitigate undesirable downstream consequences, it is critical that we understand when and how harm might be introduced throughout the ML life cycle. In this paper, we provide a framework that identifies seven distinct potential sources of downstream harm in machine learning, spanning data collection, development, and deployment. In doing so, we aim to facilitate more productive and precise communication around these issues, as well as more direct, application-grounded ways to mitigate them.

References in corpus (18)

• SMOTE: Synthetic Minority Over-sampling Technique
• Model Cards for Model Reporting
• Equality of Opportunity in Supervised Learning
• Word Embeddings Quantify 100 Years of Gender and Ethnic Stereotypes
• Counterfactual Fairness
• Mitigating Bias in Algorithmic Hiring: Evaluating Claims and Practices
• The Measure and Mismeasure of Fairness
• Bias in Bios: A Case Study of Semantic Representation Bias in a High-Stakes Setting
• On the (im)possibility of fairness
• A Convex Framework for Fair Regression
• No Classification without Representation: Assessing Geodiversity Issues in Open Data Sets for the Developing World
• On Formalizing Fairness in Prediction with Machine Learning
• InclusiveFaceNet: Improving Face Attribute Detection with Race and Gender Diversity
• Differential Privacy Has Disparate Impact on Model Accuracy
• Learning Tasks for Multitask Learning: Heterogenous Patient Populations in the ICU
• Bringing the People Back In: Contesting Benchmark Machine Learning Datasets
• Characterising Bias in Compressed Models
• Probably Approximately Metric-Fair Learning

Cited by in corpus (22)

• Fairness in Machine Learning: A Survey
• Unpacking the Expressed Consequences of AI Research in Broader Impact Statements
• Designing Deep Reinforcement Learning for Human Parameter Exploration
• Dos and Don'ts of Machine Learning in Computer Security
• Can Fairness be Automated? Guidelines and Opportunities for Fairness-aware AutoML
• Directional Bias Amplification
• How Costly is Noise? Data and Disparities in Consumer Credit
• Responsible AI: Gender bias assessment in emotion recognition
• Counterfactual fairness: removing direct effects through regularization
• Representation Matters: Assessing the Importance of Subgroup Allocations in Training Data
• Neural Temporal Point Processes For Modelling Electronic Health Records
• To Split or Not to Split: The Impact of Disparate Treatment in Classification
• Domain Adaptive Decision Trees: Implications for Accuracy and Fairness
• Jigsaw-VAE: Towards Balancing Features in Variational Autoencoders
• Representative Methods of Computational Socioeconomics
• Distributive Justice and Fairness Metrics in Automated Decision-making: How Much Overlap Is There?
• Leveraging traditional ecological knowledge in ecosystem restoration projects utilizing machine learning
• Pink for Princesses, Blue for Superheroes: The Need to Examine Gender Stereotypes in Kid's Products in Search and Recommendations
• Exploring Biases and Prejudice of Facial Synthesis via Semantic Latent Space
• Fairness-aware Summarization for Justified Decision-Making
• Towards classification parity across cohorts
• Towards an Interface Description Template for AI-enabled Systems