Subverting Fair Image Search with Generative Adversarial Perturbations

Abstract

In this work we explore the intersection fairness and robustness in the context of ranking: when a ranking model has been calibrated to achieve some definition of fairness, is it possible for an external adversary to make the ranking model behave unfairly without having access to the model or training data? To investigate this question, we present a case study in which we develop and then attack a state-of-the-art, fairness-aware image search engine using images that have been maliciously modified using a Generative Adversarial Perturbation (GAP) model. These perturbations attempt to cause the fair re-ranking algorithm to unfairly boost the rank of images containing people from an adversary-selected subpopulation. We present results from extensive experiments demonstrating that our attacks can successfully confer significant unfair advantage to people from the majority class relative to fairly-ranked baseline search results. We demonstrate that our attacks are robust across a number of variables, that they have close to zero impact on the relevance of search results, and that they succeed under a strict threat model. Our findings highlight the danger of deploying fair machine learning algorithms in-the-wild when (1) the data necessary to achieve fairness may be adversarially manipulated, and (2) the models themselves are not robust against attacks.

arXiv commentAccepted as a full paper at the 2022 ACM Conference on Fairness, Accountability, and Transparency (FAccT 22)

References in corpus (27)

• Explaining and Harnessing Adversarial Examples
• Intriguing properties of neural networks
• ViLBERT: Pretraining Task-Agnostic Visiolinguistic Representations for Vision-and-Language Tasks
• Word Embeddings Quantify 100 Years of Gender and Ethnic Stereotypes
• Delving into Transferable Adversarial Examples and Black-box Attacks
• Fairness of Exposure in Rankings
• FA*IR: A Fair Top-k Ranking Algorithm
• The Space of Transferable Adversarial Examples
• Adversarial Personalized Ranking for Recommendation
• Technical Report on the CleverHans v2.1.0 Adversarial Examples Library
• Fairness-Aware Ranking in Search & Recommendation Systems with Application to LinkedIn Talent Search
• Controlling Fairness and Bias in Dynamic Learning-to-Rank
• A Convex Framework for Fair Regression
• Closing the AI Accountability Gap: Defining an End-to-End Framework for Internal Algorithmic Auditing
• Adversarial Patch
• Causal Reasoning for Algorithmic Fairness
• Fair Regression: Quantitative Definitions and Reduction-based Algorithms
• LiFT: A Scalable Framework for Measuring Fairness in ML Applications
• Personalizing Image Search Results on Flickr
• Awareness in Practice: Tensions in Access to Sensitive Attribute Data for Antidiscrimination
• On Adversarial Bias and the Robustness of Fair Machine Learning
• What's in a Name? Reducing Bias in Bios without Access to Protected Attributes
• Fairness Through Robustness: Investigating Robustness Disparity in Deep Learning
• When Fair Ranking Meets Uncertain Inference
• Public Sphere 2.0: Targeted Commenting in Online News Media
• One word at a time: adversarial attacks on retrieval models
• Measuring Discrepancies in Airbnb Guest Acceptance Rates Using Anonymized Demographic Data

Cited by in corpus (3)

• Beyond Behaviorist Representational Harms: A Plan for Measurement and Mitigation
• When Fair Classification Meets Noisy Protected Attributes
• DSAP: Analyzing Bias Through Demographic Comparison of Datasets