No Classification without Representation: Assessing Geodiversity Issues in Open Data Sets for the Developing World
arXiv:1711.08536
Abstract
Modern machine learning systems such as image classifiers rely heavily on large scale data sets for training. Such data sets are costly to create, thus in practice a small number of freely available, open source data sets are widely used. We suggest that examining the geo-diversity of open data sets is critical before adopting a data set for use cases in the developing world. We analyze two large, publicly available image data sets to assess geo-diversity and find that these data sets appear to exhibit an observable amerocentric and eurocentric representation bias. Further, we analyze classifiers trained on these data sets to assess the impact of these training distributions and find strong differences in the relative performance on images from different locales. These results emphasize the need to ensure geo-representation when constructing data sets for use in the developing world.
Presented at NIPS 2017 Workshop on Machine Learning for the Developing World
References in corpus (3)
Cited by in corpus (51)
- WILDS: A Benchmark of in-the-Wild Distribution Shifts
- Measuring Robustness to Natural Distribution Shifts in Image Classification
- FairFace: Face Attribute Dataset for Balanced Race, Gender, and Age
- The Butterfly Effect in Artificial Intelligence Systems: Implications for AI Bias and Fairness
- Federated Learning with Matched Averaging
- Does Object Recognition Work for Everyone?
- A Framework for Understanding Sources of Harm throughout the Machine Learning Life Cycle
- A Study of Face Obfuscation in ImageNet
- Learning Tasks for Multitask Learning: Heterogenous Patient Populations in the ICU
- Towards Accountability for Machine Learning Datasets: Practices from Software Engineering and Infrastructure
- Characterising Bias in Compressed Models
- Large image datasets: A pyrrhic win for computer vision?
- Are All Training Examples Created Equal? An Empirical Study
- Data Representativeness in Accessibility Datasets: A Meta-Analysis
- Gender and Racial Bias in Visual Question Answering Datasets
- Out of Context: Investigating the Bias and Fairness Concerns of "Artificial Intelligence as a Service"
- Social Inclusion in Curated Contexts: Insights from Museum Practices
- Better, Not Just More: Data-Centric Machine Learning for Earth Observation
- What are the biases in my word embedding?
- The Limits of Global Inclusion in AI Development
- MLM: A Benchmark Dataset for Multitask Learning with Multiple Languages and Modalities
- Generating 3D Bio-Printable Patches Using Wound Segmentation and Reconstruction to Treat Diabetic Foot Ulcers
- Censorship of Online Encyclopedias: Implications for NLP Models
- Responsible AI: Gender bias assessment in emotion recognition
- Flickr Africa: Examining Geo-Diversity in Large-Scale, Human-Centric Visual Data
- Mind the GAP: A Balanced Corpus of Gendered Ambiguous Pronouns
- Representation Matters: Assessing the Importance of Subgroup Allocations in Training Data
- Listen and Speak Fairly: A Study on Semantic Gender Bias in Speech Integrated Large Language Models
- LandCoverNet: A global benchmark land cover classification training dataset
- Does enforcing fairness mitigate biases caused by subpopulation shift?
- Seeing is Knowing! Fact-based Visual Question Answering using Knowledge Graph Embeddings
- Representative Methods of Computational Socioeconomics
- Assessment of the Reliablity of a Model's Decision by Generalizing Attribution to the Wavelet Domain
- A Framework for Challenge Design: Insight and Deployment Challenges to Address Medical Image Analysis Problems
- Metadata Normalization
- Leveraging traditional ecological knowledge in ecosystem restoration projects utilizing machine learning
- Uncovering the Bias in Facial Expressions
- Towards a Measure of Individual Fairness for Deep Learning
- Automated Cleanup of the ImageNet Dataset by Model Consensus, Explainability and Confident Learning
- Exploring Data Pipelines through the Process Lens: a Reference Model forComputer Vision
- Exploring Biases and Prejudice of Facial Synthesis via Semantic Latent Space
- Adapting Convolutional Neural Networks for Geographical Domain Shift
- Teaching a GAN What Not to Learn
- An Information Retrieval Approach to Building Datasets for Hate Speech Detection
- Generating a Training Dataset for Land Cover Classification to Advance Global Development
- Broaden the Vision: Geo-Diverse Visual Commonsense Reasoning
- Towards Auditability for Fairness in Deep Learning
- CRAFT: Cultural Russian-Oriented Dataset Adaptation for Focused Text-to-Image Generation
- Ethically aligned Deep Learning: Unbiased Facial Aesthetic Prediction
- Understanding Gender and Racial Disparities in Image Recognition Models
- Human Imperceptible Attacks and Applications to Improve Fairness