Toward Operationalizing Pipeline-aware ML Fairness: A Research Agenda for Developing Practical Guidelines and Tools

Abstract

While algorithmic fairness is a thriving area of research, in practice, mitigating issues of bias often gets reduced to enforcing an arbitrarily chosen fairness metric, either by enforcing fairness constraints during the optimization step, post-processing model outputs, or by manipulating the training data. Recent work has called on the ML community to take a more holistic approach to tackle fairness issues by systematically investigating the many design choices made through the ML pipeline, and identifying interventions that target the issue's root cause, as opposed to its symptoms. While we share the conviction that this pipeline-based approach is the most appropriate for combating algorithmic unfairness on the ground, we believe there are currently very few methods of \emph{operationalizing} this approach in practice. Drawing on our experience as educators and practitioners, we first demonstrate that without clear guidelines and toolkits, even individuals with specialized ML knowledge find it challenging to hypothesize how various design choices influence model behavior. We then consult the fair-ML literature to understand the progress to date toward operationalizing the pipeline-aware approach: we systematically collect and organize the prior work that attempts to detect, measure, and mitigate various sources of unfairness through the ML pipeline. We utilize this extensive categorization of previous contributions to sketch a research agenda for the community. We hope this work serves as the stepping stone toward a more comprehensive set of resources for ML researchers, practitioners, and students interested in exploring, designing, and testing pipeline-oriented approaches to algorithmic fairness.

EAAMO'23 (Archival)

References in corpus (64)

• Equality of Opportunity in Supervised Learning
• Improving fairness in machine learning systems: What do industry practitioners need?
• Underspecification Presents Challenges for Credibility in Modern Machine Learning
• The Fallacy of AI Functionality
• Towards Fairer Datasets: Filtering and Balancing the Distribution of the People Subtree in the ImageNet Hierarchy
• Problem Formulation and Fairness
• Avoiding Discrimination through Causal Reasoning
• A Case for Humans-in-the-Loop: Decisions in the Presence of Erroneous Algorithmic Scores
• Towards Understanding and Mitigating Social Biases in Language Models
• From Parity to Preference-based Notions of Fairness in Classification
• Fair Preprocessing: Towards Understanding Compositional Fairness of Data Transformers in Machine Learning Pipeline
• Gender Bias in Word Embeddings: A Comprehensive Analysis of Frequency, Syntax, and Semantics
• Fair Regression: Quantitative Definitions and Reduction-based Algorithms
• Learning with Noisy Labels Revisited: A Study Using Real-World Human Annotations
• Towards Equity and Algorithmic Fairness in Student Grade Prediction
• Fairness in Risk Assessment Instruments: Post-Processing to Achieve Counterfactual Equalized Odds
• FairFil: Contrastive Neural Debiasing Method for Pretrained Text Encoders
• Amazon SageMaker Clarify: Machine Learning Bias Detection and Explainability in the Cloud
• Marrying Fairness and Explainability in Supervised Learning
• The effect of differential victim crime reporting on predictive policing systems
• FairBatch: Batch Selection for Model Fairness
• Fair Classification with Group-Dependent Label Noise
• Understanding the Representation and Representativeness of Age in AI Data Sets
• The Fairness of Risk Scores Beyond Classification: Bipartite Ranking and the xAUC Metric
• De-biasing "bias" measurement
• Learning the Pareto Front with Hypernetworks
• AutoBalance: Optimized Loss Functions for Imbalanced Data
• Learning Certified Individually Fair Representations
• Fairness via Representation Neutralization
• Subgroup Generalization and Fairness of Graph Neural Networks
• Between Subjectivity and Imposition: Power Dynamics in Data Annotation for Computer Vision
• Can I Trust My Fairness Metric? Assessing Fairness with Unlabeled Data and Bayesian Inference
• Directional Bias Amplification
• Achieving Equalized Odds by Resampling Sensitive Attributes
• DeBayes: a Bayesian Method for Debiasing Network Embeddings
• On Fair Selection in the Presence of Implicit and Differential Variance
• Refining Language Models with Compositional Explanations
• FairCanary: Rapid Continuous Explainable Fairness
• Affirmative Algorithms: The Legal Grounds for Fairness as Awareness
• SenSeI: Sensitive Set Invariance for Enforcing Individual Fairness
• FairEGM: Fair Link Prediction and Recommendation via Emulated Graph Modification
• Accounting for Model Uncertainty in Algorithmic Discrimination
• Loss-Aversively Fair Classification
• Group-Fair Online Allocation in Continuous Time
• Assessing Fairness in the Presence of Missing Data
• Censorship of Online Encyclopedias: Implications for NLP Models
• Chasing Your Long Tails: Differentially Private Prediction in Health Care Settings
• The Rich Get Richer: Disparate Impact of Semi-Supervised Learning
• Fairness-aware Model-agnostic Positive and Unlabeled Learning
• Matching Learned Causal Effects of Neural Networks with Domain Priors
• Incorporating Interpretable Output Constraints in Bayesian Neural Networks
• Does enforcing fairness mitigate biases caused by subpopulation shift?
• Is Fairness Only Metric Deep? Evaluating and Addressing Subgroup Gaps in Deep Metric Learning
• Conditional Contrastive Learning with Kernel
• Fair Representation Learning through Implicit Path Alignment
• Understanding Instance-Level Impact of Fairness Constraints
• An Axiomatic Theory of Provably-Fair Welfare-Centric Machine Learning
• Learning fair representation with a parametric integral probability metric
• Active Fairness Auditing
• Controlling Directions Orthogonal to a Classifier
• Can Information Flows Suggest Targets for Interventions in Neural Circuits?
• A Sandbox Tool to Bias(Stress)-Test Fairness Algorithms
• Can Less be More? When Increasing-to-Balancing Label Noise Rates Considered Beneficial
• Contrastive Mixture of Posteriors for Counterfactual Inference, Data Integration and Fairness

Cited by in corpus (3)

• A Scoping Study of Evaluation Practices for Responsible AI Tools: Steps Towards Effectiveness Evaluations
• The "Who", "What", and "How" of Responsible AI Governance: A Systematic Review and Meta-Analysis of (Actor, Stage)-Specific Tools
• Allocation Multiplicity: Evaluating the Promises of the Rashomon Set