31 citations · 59 across the 10 of their papers we have counts for
10 papers
Quantifying and mitigating the impact of label errors on model disparity metrics
Julius Adebayo, Melissa Hall, Bowen Yu +1
Errors in labels obtained via human annotation adversely affect a model's performance. Existing approaches propose ways to mitigate the effect of label error on a model's downstrea…
FACET: Fairness in Computer Vision Evaluation Benchmark
Laura Gustafson, Chloe Rolland, Nikhila Ravi +5
Computer vision models have known performance disparities across attributes such as gender and skin tone. This means during tasks such as classification and detection, model perfor…
DIG In: Evaluating Disparities in Image Generations with Indicators for Geographic Diversity
Melissa Hall, Candace Ross, Adina Williams +3
The unprecedented photorealistic results achieved by recent text-to-image generative systems and their increasing use as plug-and-play content creation solutions make it crucial to…
Pinpointing Why Object Recognition Performance Degrades Across Income Levels and Geographies
Laura Gustafson, Megan Richards, Melissa Hall +3
Despite impressive advances in object-recognition, deep learning systems' performance degrades significantly across geographies and lower income levels raising pressing concerns of…
Towards Reliable Assessments of Demographic Disparities in Multi-Label Image Classifiers
Melissa Hall, Bobbie Chern, Laura Gustafson +4
Disaggregated performance metrics across demographic groups are a hallmark of fairness assessments in computer vision. These metrics successfully incentivized performance improveme…
Vision-Language Models Performing Zero-Shot Tasks Exhibit Gender-based Disparities
Melissa Hall, Laura Gustafson, Aaron Adcock +2
We explore the extent to which zero-shot vision-language models exhibit gender bias for different vision tasks. Vision models traditionally required task-specific labels for repres…
"I'm sorry to hear that": Finding New Biases in Language Models with a Holistic Descriptor Dataset
Eric Michael Smith, Melissa Hall, Melanie Kambadur +2
As language models grow in popularity, it becomes increasingly important to clearly measure all possible markers of demographic identity in order to avoid perpetuating existing soc…
Understanding out-of-distribution accuracies through quantifying difficulty of test samples
Berfin Simsek, Melissa Hall, Levent Sagun
Existing works show that although modern neural networks achieve remarkable generalization performance on the in-distribution (ID) dataset, the accuracy drops significantly on the…
A Systematic Study of Bias Amplification
Melissa Hall, Laurens van der Maaten, Laura Gustafson +2
Recent research suggests that predictions made by machine-learning models can amplify biases present in the training data. When a model amplifies bias, it makes certain predictions…
Fairness On The Ground: Applying Algorithmic Fairness Approaches to Production Systems
Chloé Bakalar, Renata Barreto, Stevie Bergman +13
Many technical approaches have been proposed for ensuring that decisions made by machine learning systems are fair, but few of these proposals have been stress-tested in real-world…