Toward Operationalizing Pipeline-aware ML Fairness: A Research Agenda for Developing Practical Guidelines and Tools

Abstract

While algorithmic fairness is a thriving area of research, in practice, mitigating issues of bias often gets reduced to enforcing an arbitrarily chosen fairness metric, either by enforcing fairness constraints during the optimization step, post-processing model outputs, or by manipulating the training data. Recent work has called on the ML community to take a more holistic approach to tackle fairness issues by systematically investigating the many design choices made through the ML pipeline, and identifying interventions that target the issue's root cause, as opposed to its symptoms. While we share the conviction that this pipeline-based approach is the most appropriate for combating algorithmic unfairness on the ground, we believe there are currently very few methods of \emph{operationalizing} this approach in practice. Drawing on our experience as educators and practitioners, we first demonstrate that without clear guidelines and toolkits, even individuals with specialized ML knowledge find it challenging to hypothesize how various design choices influence model behavior. We then consult the fair-ML literature to understand the progress to date toward operationalizing the pipeline-aware approach: we systematically collect and organize the prior work that attempts to detect, measure, and mitigate various sources of unfairness through the ML pipeline. We utilize this extensive categorization of previous contributions to sketch a research agenda for the community. We hope this work serves as the stepping stone toward a more comprehensive set of resources for ML researchers, practitioners, and students interested in exploring, designing, and testing pipeline-oriented approaches to algorithmic fairness.

arXiv commentEAAMO'23 (Archival)

References in corpus (83)

• Model Cards for Model Reporting
• Equality of Opportunity in Supervised Learning
• Counterfactual Fairness
• Improving fairness in machine learning systems: What do industry practitioners need?
• Underspecification Presents Challenges for Credibility in Modern Machine Learning
• Preventing Fairness Gerrymandering: Auditing and Learning for Subgroup Fairness
• The Fallacy of AI Functionality
• Measurement and Fairness
• Towards Fairer Datasets: Filtering and Balancing the Distribution of the People Subtree in the ImageNet Hierarchy
• Problem Formulation and Fairness
• Fairness Without Demographics in Repeated Loss Minimization
• CERTIFAI: Counterfactual Explanations for Robustness, Transparency, Interpretability, and Fairness of Artificial Intelligence models
• Avoiding Discrimination through Causal Reasoning
• A Case for Humans-in-the-Loop: Decisions in the Presence of Erroneous Algorithmic Scores
• Beyond Parity: Fairness Objectives for Collaborative Filtering
• Why Is My Classifier Discriminatory?
• Fairness Under Unawareness: Assessing Disparity When Protected Class Is Unobserved
• Towards Understanding and Mitigating Social Biases in Language Models
• From Parity to Preference-based Notions of Fairness in Classification
• Invariant Representations without Adversarial Training
• Fair Preprocessing: Towards Understanding Compositional Fairness of Data Transformers in Machine Learning Pipeline
• Residual Unfairness in Fair Machine Learning from Prejudiced Data
• Gender Bias in Word Embeddings: A Comprehensive Analysis of Frequency, Syntax, and Semantics
• Fair Regression: Quantitative Definitions and Reduction-based Algorithms
• Learning with Noisy Labels Revisited: A Study Using Real-World Human Annotations
• Towards Equity and Algorithmic Fairness in Student Grade Prediction
• FairFil: Contrastive Neural Debiasing Method for Pretrained Text Encoders
• Fairness in Risk Assessment Instruments: Post-Processing to Achieve Counterfactual Equalized Odds
• Training individually fair ML models with Sensitive Subspace Robustness
• Amazon SageMaker Clarify: Machine Learning Bias Detection and Explainability in the Cloud
• Marrying Fairness and Explainability in Supervised Learning
• Fairness Violations and Mitigation under Covariate Shift
• The effect of differential victim crime reporting on predictive policing systems
• FairBatch: Batch Selection for Model Fairness
• Probably Approximately Metric-Fair Learning
• Data preprocessing to mitigate bias: A maximum entropy based approach
• Fair Classification with Group-Dependent Label Noise
• Understanding the Representation and Representativeness of Age in AI Data Sets
• The Fairness of Risk Scores Beyond Classification: Bipartite Ranking and the xAUC Metric
• Classification with Fairness Constraints: A Meta-Algorithm with Provable Guarantees
• Noise-tolerant fair classification
• De-biasing "bias" measurement
• Learning the Pareto Front with Hypernetworks
• AutoBalance: Optimized Loss Functions for Imbalanced Data
• Learning Certified Individually Fair Representations
• Fairness via Representation Neutralization
• Average Individual Fairness: Algorithms, Generalization and Experiments
• Subgroup Generalization and Fairness of Graph Neural Networks
• Between Subjectivity and Imposition: Power Dynamics in Data Annotation for Computer Vision
• Can I Trust My Fairness Metric? Assessing Fairness with Unlabeled Data and Bayesian Inference
• Directional Bias Amplification
• Achieving Equalized Odds by Resampling Sensitive Attributes
• DeBayes: a Bayesian Method for Debiasing Network Embeddings
• On Fair Selection in the Presence of Implicit and Differential Variance
• FairCanary: Rapid Continuous Explainable Fairness
• Refining Language Models with Compositional Explanations
• Affirmative Algorithms: The Legal Grounds for Fairness as Awareness
• SenSeI: Sensitive Set Invariance for Enforcing Individual Fairness
• Envy-Free Classification
• FairEGM: Fair Link Prediction and Recommendation via Emulated Graph Modification
• Accounting for Model Uncertainty in Algorithmic Discrimination
• Loss-Aversively Fair Classification
• Group-Fair Online Allocation in Continuous Time
• Assessing Fairness in the Presence of Missing Data
• Censorship of Online Encyclopedias: Implications for NLP Models
• Chasing Your Long Tails: Differentially Private Prediction in Health Care Settings
• Fairness-aware Model-agnostic Positive and Unlabeled Learning
• The Rich Get Richer: Disparate Impact of Semi-Supervised Learning
• Matching Learned Causal Effects of Neural Networks with Domain Priors
• Does enforcing fairness mitigate biases caused by subpopulation shift?
• Incorporating Interpretable Output Constraints in Bayesian Neural Networks
• Is Fairness Only Metric Deep? Evaluating and Addressing Subgroup Gaps in Deep Metric Learning
• Fair Representation Learning through Implicit Path Alignment
• Conditional Contrastive Learning with Kernel
• Understanding Instance-Level Impact of Fairness Constraints
• Learning fair representation with a parametric integral probability metric
• An Axiomatic Theory of Provably-Fair Welfare-Centric Machine Learning
• Can Information Flows Suggest Targets for Interventions in Neural Circuits?
• Can Less be More? When Increasing-to-Balancing Label Noise Rates Considered Beneficial
• A Sandbox Tool to Bias(Stress)-Test Fairness Algorithms
• Controlling Directions Orthogonal to a Classifier
• Active Fairness Auditing
• Contrastive Mixture of Posteriors for Counterfactual Inference, Data Integration and Fairness

Cited by in corpus (3)

• A Scoping Study of Evaluation Practices for Responsible AI Tools: Steps Towards Effectiveness Evaluations
• The "Who", "What", and "How" of Responsible AI Governance: A Systematic Review and Meta-Analysis of (Actor, Stage)-Specific Tools
• Allocation Multiplicity: Evaluating the Promises of the Rashomon Set