Assessing and Remedying Coverage for a Given Dataset
arXiv:1810.06742 · doi:10.1109/ICDE.2019.00056
Abstract
Data analysis impacts virtually every aspect of our society today. Often, this analysis is performed on an existing dataset, possibly collected through a process that the data scientists had limited control over. The existing data analyzed may not include the complete universe, but it is expected to cover the diversity of items in the universe. Lack of adequate coverage in the dataset can result in undesirable outcomes such as biased decisions and algorithmic racism, as well as creating vulnerabilities such as opening up room for adversarial attacks. In this paper, we assess the coverage of a given dataset over multiple categorical attributes. We first provide efficient techniques for traversing the combinatorial explosion of value combinations to identify any regions of attribute space not adequately covered by the data. Then, we determine the least amount of additional data that must be obtained to resolve this lack of adequate coverage. We confirm the value of our proposal through both theoretical analyses and comprehensive experiments on real data.
in ICDE 2019
References in corpus (6)
Cited by in corpus (11)
- Representation Bias in Data: A Survey on Identification and Resolution Techniques
- Assessing and Remedying Coverage for a Given Dataset
- Diversity and Inclusion Metrics in Subset Selection
- Dataset search: a survey
- Chameleon: Foundation Models for Fairness-aware Multi-modal Data Augmentation to Enhance Coverage of Minorities
- Towards Assessing Data Bias in Clinical Trials
- Using Constraints to Discover Sparse and Alternative Subgroup Descriptions
- Varif.ai to Vary and Verify User-Driven Diversity in Scalable Image Generation
- OmniFair: A Declarative System for Model-Agnostic Group Fairness in Machine Learning
- Patterns Count-Based Labels for Datasets
- Slice Tuner: A Selective Data Acquisition Framework for Accurate and Fair Machine Learning Models