A survey on datasets for fairness-aware machine learning

Abstract

As decision-making increasingly relies on Machine Learning (ML) and (big) data, the issue of fairness in data-driven Artificial Intelligence (AI) systems is receiving increasing attention from both research and industry. A large variety of fairness-aware machine learning solutions have been proposed which involve fairness-related interventions in the data, learning algorithms and/or model outputs. However, a vital part of proposing new approaches is evaluating them empirically on benchmark datasets that represent realistic and diverse settings. Therefore, in this paper, we overview real-world datasets used for fairness-aware machine learning. We focus on tabular data as the most common data representation for fairness-aware machine learning. We start our analysis by identifying relationships between the different attributes, particularly w.r.t. protected attributes and class attribute, using a Bayesian network. For a deeper understanding of bias in the datasets, we investigate the interesting relationships using exploratory analysis.

arXiv comment56 pages, 36 figures, 20 tables

References in corpus (33)

• SMOTE: Synthetic Minority Over-sampling Technique
• Equality of Opportunity in Supervised Learning
• Counterfactual Fairness
• Fairness Beyond Disparate Treatment & Disparate Impact: Learning Classification without Disparate Mistreatment
• Fairness in Machine Learning: A Survey
• Preventing Fairness Gerrymandering: Auditing and Learning for Subgroup Fairness
• Time-to-Event Prediction with Neural Networks and Cox Regression
• A Convex Framework for Fair Regression
• FairSight: Visual Analytics for Fairness in Decision Making
• Fairway: A Way to Build Fair ML Software
• On the relation between accuracy and fairness in binary classification
• Operationalizing Individual Fairness with Pairwise Fair Representations
• Penalizing Unfairness in Binary Classification
• AdaFair: Cumulative Fairness Adaptive Boosting
• Minimax Pareto Fairness: A Multi Objective Perspective
• How fair can we go in machine learning? Assessing the boundaries of fairness in decision trees
• Fair clustering via equitable group representations
• Fairness Behind a Veil of Ignorance: A Welfare Analysis for Automated Decision Making
• Bias In, Bias Out? Evaluating the Folk Wisdom
• Identifying Significant Predictive Bias in Classifiers
• Fairer and more accurate, but for whom?
• Individual Fairness for $k$-Clustering
• Making Fair ML Software using Trustworthy Explanation
• Average Individual Fairness: Algorithms, Generalization and Experiments
• Algorithmic Decision Making with Conditional Fairness
• Fairness in Clustering with Multiple Sensitive Attributes
• Consequence-aware Sequential Counterfactual Generation
• Hunting for Discriminatory Proxies in Linear Regression Models
• Fairness with Overlapping Groups
• Distributional Individual Fairness in Clustering
• Pairwise Fairness for Ranking and Regression
• Fair Outlier Detection
• Probabilistic Fair Clustering

Cited by in corpus (2)

• Fair Classification via Domain Adaptation: A Dual Adversarial Learning Approach
• Towards Feasible Counterfactual Explanations: A Taxonomy Guided Template-based NLG Method