A Framework for Understanding Sources of Harm throughout the Machine Learning Life Cycle

Abstract

As machine learning (ML) increasingly affects people and society, awareness of its potential unwanted consequences has also grown. To anticipate, prevent, and mitigate undesirable downstream consequences, it is critical that we understand when and how harm might be introduced throughout the ML life cycle. In this paper, we provide a framework that identifies seven distinct potential sources of downstream harm in machine learning, spanning data collection, development, and deployment. In doing so, we aim to facilitate more productive and precise communication around these issues, as well as more direct, application-grounded ways to mitigate them.

References in corpus (9)

• Equality of Opportunity in Supervised Learning
• Mitigating Bias in Algorithmic Hiring: Evaluating Claims and Practices
• Bias in Bios: A Case Study of Semantic Representation Bias in a High-Stakes Setting
• On the (im)possibility of fairness
• A Convex Framework for Fair Regression
• No Classification without Representation: Assessing Geodiversity Issues in Open Data Sets for the Developing World
• Differential Privacy Has Disparate Impact on Model Accuracy
• Bringing the People Back In: Contesting Benchmark Machine Learning Datasets
• Characterising Bias in Compressed Models

Cited by in corpus (22)

• Fairness in Machine Learning: A Survey
• Unpacking the Expressed Consequences of AI Research in Broader Impact Statements
• Designing Deep Reinforcement Learning for Human Parameter Exploration
• Dos and Don'ts of Machine Learning in Computer Security
• Can Fairness be Automated? Guidelines and Opportunities for Fairness-aware AutoML
• Directional Bias Amplification
• How Costly is Noise? Data and Disparities in Consumer Credit
• Responsible AI: Gender bias assessment in emotion recognition
• Counterfactual fairness: removing direct effects through regularization
• Representation Matters: Assessing the Importance of Subgroup Allocations in Training Data
• Neural Temporal Point Processes For Modelling Electronic Health Records
• To Split or Not to Split: The Impact of Disparate Treatment in Classification
• Domain Adaptive Decision Trees: Implications for Accuracy and Fairness
• Jigsaw-VAE: Towards Balancing Features in Variational Autoencoders
• Representative Methods of Computational Socioeconomics
• Distributive Justice and Fairness Metrics in Automated Decision-making: How Much Overlap Is There?
• Leveraging traditional ecological knowledge in ecosystem restoration projects utilizing machine learning
• Pink for Princesses, Blue for Superheroes: The Need to Examine Gender Stereotypes in Kid's Products in Search and Recommendations
• Exploring Biases and Prejudice of Facial Synthesis via Semantic Latent Space
• Fairness-aware Summarization for Justified Decision-Making
• Towards classification parity across cohorts
• Towards an Interface Description Template for AI-enabled Systems