Does Object Recognition Work for Everyone?
arXiv:1906.02659
Abstract
The paper analyzes the accuracy of publicly available object-recognition systems on a geographically diverse dataset. This dataset contains household items and was designed to have a more representative geographical coverage than commonly used image datasets in object recognition. We find that the systems perform relatively poorly on household items that commonly occur in countries with a low household income. Qualitative analyses suggest the drop in performance is primarily due to appearance differences within an object class (e.g., dish soap) and due to items appearing in a different context (e.g., toothbrushes appearing outside of bathrooms). The results of our study suggest that further work is needed to make object-recognition systems work equally well for people across different countries and income levels.
References in corpus (8)
- Equality of Opportunity in Supervised Learning
- The Open Images Dataset V4: Unified image classification, object detection, and visual relationship detection at scale
- On Fairness and Calibration
- Word Translation Without Parallel Data
- No Classification without Representation: Assessing Geodiversity Issues in Open Data Sets for the Developing World
- Identifying and Correcting Label Bias in Machine Learning
- Tencent ML-Images: A Large-Scale Multi-Label Image Database for Visual Representation Learning
- ConvNets and ImageNet Beyond Accuracy: Understanding Mistakes and Uncovering Biases
Cited by in corpus (12)
- DINOv2: Learning Robust Visual Features without Supervision
- Ethics-Based Auditing of Automated Decision-Making Systems: Intervention Points and Policy Implications
- Characterising Bias in Compressed Models
- Data Representativeness in Accessibility Datasets: A Meta-Analysis
- Out of Context: Investigating the Bias and Fairness Concerns of "Artificial Intelligence as a Service"
- Perfect density models cannot guarantee anomaly detection
- Angler: Helping Machine Translation Practitioners Prioritize Model Improvements
- Toward Fairness in Speech Recognition: Discovery and mitigation of performance disparities
- The Limits of Global Inclusion in AI Development
- Flickr Africa: Examining Geo-Diversity in Large-Scale, Human-Centric Visual Data
- Distribution-aware Fairness Test Generation
- Teaching a GAN What Not to Learn