Algorithmic Fairness Datasets: the Story so Far

Abstract

Data-driven algorithms are studied in diverse domains to support critical decisions, directly impacting people's well-being. As a result, a growing community of researchers has been investigating the equity of existing algorithms and proposing novel ones, advancing the understanding of risks and opportunities of automated decision-making for historically disadvantaged populations. Progress in fair Machine Learning hinges on data, which can be appropriately used only if adequately documented. Unfortunately, the algorithmic fairness community suffers from a collective data documentation debt caused by a lack of information on specific resources (opacity) and scatteredness of available information (sparsity). In this work, we target data documentation debt by surveying over two hundred datasets employed in algorithmic fairness research, and producing standardized and searchable documentation for each of them. Moreover we rigorously identify the three most popular fairness datasets, namely Adult, COMPAS and German Credit, for which we compile in-depth documentation. This unifying documentation effort supports multiple contributions. Firstly, we summarize the merits and limitations of Adult, COMPAS and German Credit, adding to and unifying recent scholarship, calling into question their suitability as general-purpose fairness benchmarks. Secondly, we document and summarize hundreds of available alternatives, annotating their domain and supported fairness tasks, along with additional properties of interest for fairness researchers. Finally, we analyze these datasets from the perspective of five important data curation topics: anonymization, consent, inclusivity, sensitive attributes, and transparency. We discuss different approaches and levels of attention to these topics, making them tangible, and distill them into a set of best practices for the curation of novel resources.

arXiv commentPublished in Data Mining and Knowledge Discovery https://doi.org/10.1007/s10618-022-00854-z

References in corpus (45)

• The HAM10000 dataset, a large collection of multi-source dermatoscopic images of common pigmented skin lesions
• Semantics derived automatically from language corpora contain human-like biases
• Ups and Downs: Modeling the Visual Evolution of Fashion Trends with One-Class Collaborative Filtering
• Improving fairness in machine learning systems: What do industry practitioners need?
• Fairness Beyond Disparate Treatment & Disparate Impact: Learning Classification without Disparate Mistreatment
• Contact patterns in a high school: a comparison between data collected using wearable sensors, contact diaries and friendship surveys
• Fairness of Exposure in Rankings
• Learning Deep Representation for Face Alignment with Auxiliary Attributes
• Translation-based Recommendation
• Comparing Community Structure to Characteristics in Online Collegiate Social Networks
• Bias in Bios: A Case Study of Semantic Representation Bias in a High-Stakes Setting
• Lessons from Archives: Strategies for Collecting Sociocultural Data in Machine Learning
• Towards Fairer Datasets: Filtering and Balancing the Distribution of the People Subtree in the ImageNet Hierarchy
• BOLD: Dataset and Metrics for Measuring Biases in Open-Ended Language Generation
• Learning from History and Present: Next-item Recommendation via Discriminatively Exploiting User Behaviors
• A Unified Approach to Quantifying Algorithmic Unfairness: Measuring Individual & Group Unfairness via Inequality Indices
• CERTIFAI: Counterfactual Explanations for Robustness, Transparency, Interpretability, and Fairness of Artificial Intelligence models
• Image Representations Learned With Unsupervised Pre-Training Contain Human-like Biases
• Fairness Under Unawareness: Assessing Disparity When Protected Class Is Unobserved
• Quantifying Search Bias: Investigating Sources of Bias for Political Searches in Social Media
• Detecting Emergent Intersectional Biases: Contextualized Word Embeddings Contain a Distribution of Human-like Biases
• Auditing Search Engines for Differential Satisfaction Across Demographics
• Disparate Impact of Artificial Intelligence Bias in Ridehailing Economy's Price Discrimination Algorithms
• FlipTest: Fairness Testing via Optimal Transport
• Towards Equity and Algorithmic Fairness in Student Grade Prediction
• Fair clustering via equitable group representations
• Fairness in Risk Assessment Instruments: Post-Processing to Achieve Counterfactual Equalized Odds
• One Label, One Billion Faces: Usage and Consistency of Racial Categories in Computer Vision
• Fairness Violations and Mitigation under Covariate Shift
• Case Study: Predictive Fairness to Reduce Misdemeanor Recidivism Through Social Service Interventions
• Fair Classification with Group-Dependent Label Noise
• A Step Toward More Inclusive People Annotations for Fairness
• Algorithmic Decision Making with Conditional Fairness
• Personality Traits and Drug Consumption. A Story Told by Data
• FairOD: Fairness-aware Outlier Detection
• Fair Allocation through Selective Information Acquisition
• Measuring Model Biases in the Absence of Ground Truth
• Artificial mental phenomena: Psychophysics as a framework to detect perception biases in AI models
• Accounting for Model Uncertainty in Algorithmic Discrimination
• Loss-Aversively Fair Classification
• Rawlsian Fair Adaptation of Deep Learning Classifiers
• GAEA: Graph Augmentation for Equitable Access via Reinforcement Learning
• Curatr: A Platform for Semantic Analysis and Curation of Historical Literary Texts
• Convex Fairness Constrained Model Using Causal Effect Estimators
• Garbage In, Garbage Out? Do Machine Learning Application Papers in Social Computing Report Where Human-Labeled Training Data Comes From?

Cited by in corpus (1)

• Measuring Fairness Under Unawareness of Sensitive Attributes: A Quantification-Based Approach