Fairness Testing: Testing Software for Discrimination

Abstract

This paper defines software fairness and discrimination and develops a testing-based method for measuring if and how much software discriminates, focusing on causality in discriminatory behavior. Evidence of software discrimination has been found in modern software systems that recommend criminal sentences, grant access to financial products, and determine who is allowed to participate in promotions. Our approach, Themis, generates efficient test suites to measure discrimination. Given a schema describing valid system inputs, Themis generates discrimination tests automatically and does not require an oracle. We evaluate Themis on 20 software systems, 12 of which come from prior work with explicit focus on avoiding discrimination. We find that (1) Themis is effective at discovering software discrimination, (2) state-of-the-art techniques for removing discrimination from algorithms fail in many situations, at times discriminating against as much as 98% of an input subdomain, (3) Themis optimizations are effective at producing efficient test suites for measuring discrimination, and (4) Themis is more efficient on systems that exhibit more discrimination. We thus demonstrate that fairness testing is a critical aspect of the software development cycle in domains with possible discrimination and provide initial tools for measuring software discrimination.

Sainyam Galhotra, Yuriy Brun, and Alexandra Meliou. 2017. Fairness Testing: Testing Software for Discrimination. In Proceedings of 2017 11th Joint Meeting of the European Software Engineering Conference and the ACM SIGSOFT Symposium on the Foundations of Software Engineering (ESEC/FSE), Paderborn, Germany, September 4-8, 2017 (ESEC/FSE'17). https://doi.org/10.1145/3106237.3106277, ESEC/FSE, 2017

References in corpus (5)

• Scikit-learn: Machine Learning in Python
• Counterfactual Fairness
• Fairness in Learning: Classic and Contextual Bandits
• Iterative Orthogonal Feature Projection for Diagnosing Bias in Black-Box Models
• QFix: Diagnosing errors through query histories

Cited by in corpus (74)

• Improving fairness in machine learning systems: What do industry practitioners need?
• Prediction-Based Decisions and Fairness: A Catalogue of Choices, Assumptions, and Definitions
• Fairness in Machine Learning: A Survey
• Fast Parallel Hypertree Decompositions in Logarithmic Recursion Depth
• AI Fairness 360: An Extensible Toolkit for Detecting, Understanding, and Mitigating Unwanted Algorithmic Bias
• Automated Directed Fairness Testing
• Bias in Machine Learning Software: Why? How? What to do?
• Algorithmic decision-making in AVs: Understanding ethical and technical concerns for smart cities
• Machine Learning Testing: Survey, Landscapes and Horizons
• Fair Preprocessing: Towards Understanding Compositional Fairness of Data Transformers in Machine Learning Pipeline
• Do the Machine Learning Models on a Crowd Sourced Platform Exhibit Bias? An Empirical Study on Model Fairness
• Value Cards: An Educational Toolkit for Teaching Social Impacts of Machine Learning through Deliberation
• FlipTest: Fairness Testing via Optimal Transport
• Machine learning fairness notions: Bridging the gap with real-world applications
• LiFT: A Scalable Framework for Measuring Fairness in ML Applications
• Fairness-aware Configuration of Machine Learning Libraries
• Causal Testing: Finding Defects' Root Causes
• Operationalizing Human Values in Software Engineering: A Survey
• Open the Black Box Data-Driven Explanation of Black Box Decision Systems
• Socially Responsible AI Algorithms: Issues, Purposes, and Challenges
• Latent Imitator: Generating Natural Individual Discriminatory Instances for Black-Box Fairness Testing
• Testing DNN Image Classifiers for Confusion & Bias Errors
• Towards Understanding Fairness and its Composition in Ensemble Machine Learning
• Making Fair ML Software using Trustworthy Explanation
• The Bouncer Problem: Challenges to Remote Explainability
• Automated Test Generation to Detect Individual Discrimination in AI Models
• FairPrep: Promoting Data to a First-Class Citizen in Studies on Fairness-Enhancing Interventions
• Ensuring Fairness Beyond the Training Data
• My Model is Unfair, Do People Even Care? Visual Design Affects Trust and Perceived Bias in Machine Learning
• Verifying Individual Fairness in Machine Learning Models
• Software Engineering for Fairness: A Case Study with Hyperparameter Optimization
• Fairness Through Robustness: Investigating Robustness Disparity in Deep Learning
• Fix Fairness, Don't Ruin Accuracy: Performance Aware Fairness Repair using AutoML
• Capuchin: Causal Database Repair for Algorithmic Fairness
• Adversarial Sampling for Fairness Testing in Deep Neural Network
• Diverse Data Selection under Fairness Constraints
• A Unified Approach for Resilience and Causal Responsibility with Integer Linear Programming (ILP) and LP Relaxations
• Removing biased data to improve fairness and accuracy
• Unmasking Gender Bias in Recommendation Systems and Enhancing Category-Aware Fairness
• MAFT: Efficient Model-Agnostic Fairness Testing for Deep Neural Networks via Zero-Order Gradient Search
• Fair-SSL: Building fair ML Software with less data
• Counterfactual Situation Testing: Uncovering Discrimination under Fairness given the Difference
• To Split or Not to Split: The Impact of Disparate Treatment in Classification
• Addressing multiple metrics of group fairness in data-driven decision making
• Fairness Testing of Deep Image Classification with Adequacy Metrics
• Fairkit, Fairkit, on the Wall, Who's the Fairest of Them All? Supporting Data Scientists in Training Fair Models
• ShapeFlow: Dynamic Shape Interpreter for TensorFlow
• Perfectly Parallel Fairness Certification of Neural Networks
• Personalized Chatbot Trustworthiness Ratings
• An Epistemic Approach to the Formal Specification of Statistical Machine Learning
• Teaching Software Engineering for AI-Enabled Systems
• Technical Challenges for Training Fair Neural Networks
• Testing Monotonicity of Machine Learning Models
• Bias Mitigation Post-processing for Individual and Group Fairness
• FairMask: Better Fairness via Model-based Rebalancing of Protected Attributes
• QEDCartographer: Automating Formal Verification Using Reward-Free Reinforcement Learning
• Metrics and methods for a systematic comparison of fairness-aware machine learning algorithms
• Fairness Mediator: Neutralize Stereotype Associations to Mitigate Bias in Large Language Models
• Probabilistic Verification of Neural Networks Against Group Fairness
• Automatic Fairness Testing of Neural Classifiers through Adversarial Sampling
• A Review of Formal Methods applied to Machine Learning
• Fairness Through Counterfactual Utilities
• MobilityMirror: Bias-Adjusted Transportation Datasets
• SOCRATES: Towards a Unified Platform for Neural Network Analysis
• Counterfactual Situation Testing: From Single to Multidimensional Discrimination
• How does Software Change?
• Probabilistic Verification of Fairness Properties via Concentration
• Lexicographically Ordered Multi-Objective Clustering
• The Impact of Considering Human Values during Requirements Engineering Activities
• Fairness-guided SMT-based Rectification of Decision Trees and Random Forests
• The Fair Game: Auditing & Debiasing AI Algorithms Over Time
• Correct-By-Construction: Certified Individual Fairness through Neural Network Training
• Data Management for Causal Algorithmic Fairness
• Towards Integrating Fairness Transparently in Industrial Applications