WEIRD FAccTs: How Western, Educated, Industrialized, Rich, and Democratic is FAccT?

Abstract

Studies conducted on Western, Educated, Industrialized, Rich, and Democratic (WEIRD) samples are considered atypical of the world's population and may not accurately represent human behavior. In this study, we aim to quantify the extent to which the ACM FAccT conference, the leading venue in exploring Artificial Intelligence (AI) systems' fairness, accountability, and transparency, relies on WEIRD samples. We collected and analyzed 128 papers published between 2018 and 2022, accounting for 30.8% of the overall proceedings published at FAccT in those years (excluding abstracts, tutorials, and papers without human-subject studies or clear country attribution for the participants). We found that 84% of the analyzed papers were exclusively based on participants from Western countries, particularly exclusively from the U.S. (63%). Only researchers who undertook the effort to collect data about local participants through interviews or surveys added diversity to an otherwise U.S.-centric view of science. Therefore, we suggest that researchers collect data from under-represented populations to obtain an inclusive worldview. To achieve this goal, scientific communities should champion data collection from such populations and enforce transparent reporting of data biases.

To appear at ACM FAccT 2023

References in corpus (5)

• Towards Fairer Datasets: Filtering and Balancing the Distribution of the People Subtree in the ImageNet Hierarchy
• Image Representations Learned With Unsupervised Pre-Training Contain Human-like Biases
• Four Years of FAccT: A Reflexive, Mixed-Methods Analysis of Research Contributions, Shortcomings, and Future Prospects
• Marrying Fairness and Explainability in Supervised Learning
• Situated Data, Situated Systems: A Methodology to Engage with Power Relations in Natural Language Processing Research

Cited by in corpus (13)

• Data Feminism for AI
• A Scoping Study of Evaluation Practices for Responsible AI Tools: Steps Towards Effectiveness Evaluations
• Metaverse Perspectives from Japan: A Participatory Speculative Design Case Study
• "You Cannot Sound Like GPT": Signs of language discrimination and resistance in computer science publishing
• One Model Many Scores: Using Multiverse Analysis to Prevent Fairness Hacking and Evaluate the Influence of Model Design Decisions
• Lazy Data Practices Harm Fairness Research
• Low-resourced Languages and Online Knowledge Repositories: A Need-Finding Study
• Laissez-Faire Harms: Algorithmic Biases in Generative Language Models
• Identities are not Interchangeable: The Problem of Overgeneralization in Fair Machine Learning
• FairComp: Workshop on Fairness and Robustness in Machine Learning for Ubiquitous Computing
• From Model Performance to Claim: How a Change of Focus in Machine Learning Replicability Can Help Bridge the Responsibility Gap
• The State of Algorithmic Fairness in Mobile Human-Computer Interaction
• Auditing LLM-Governed Social Robots with Culture-Specific Moral Gradients