Subverting Fair Image Search with Generative Adversarial Perturbations

Abstract

In this work we explore the intersection fairness and robustness in the context of ranking: when a ranking model has been calibrated to achieve some definition of fairness, is it possible for an external adversary to make the ranking model behave unfairly without having access to the model or training data? To investigate this question, we present a case study in which we develop and then attack a state-of-the-art, fairness-aware image search engine using images that have been maliciously modified using a Generative Adversarial Perturbation (GAP) model. These perturbations attempt to cause the fair re-ranking algorithm to unfairly boost the rank of images containing people from an adversary-selected subpopulation. We present results from extensive experiments demonstrating that our attacks can successfully confer significant unfair advantage to people from the majority class relative to fairly-ranked baseline search results. We demonstrate that our attacks are robust across a number of variables, that they have close to zero impact on the relevance of search results, and that they succeed under a strict threat model. Our findings highlight the danger of deploying fair machine learning algorithms in-the-wild when (1) the data necessary to achieve fairness may be adversarially manipulated, and (2) the models themselves are not robust against attacks.

Accepted as a full paper at the 2022 ACM Conference on Fairness, Accountability, and Transparency (FAccT 22)

References in corpus (14)

• Explaining and Harnessing Adversarial Examples
• Delving into Transferable Adversarial Examples and Black-box Attacks
• The Space of Transferable Adversarial Examples
• Controlling Fairness and Bias in Dynamic Learning-to-Rank
• A Convex Framework for Fair Regression
• Closing the AI Accountability Gap: Defining an End-to-End Framework for Internal Algorithmic Auditing
• Fair Regression: Quantitative Definitions and Reduction-based Algorithms
• LiFT: A Scalable Framework for Measuring Fairness in ML Applications
• Personalizing Image Search Results on Flickr
• Awareness in Practice: Tensions in Access to Sensitive Attribute Data for Antidiscrimination
• On Adversarial Bias and the Robustness of Fair Machine Learning
• What's in a Name? Reducing Bias in Bios without Access to Protected Attributes
• One word at a time: adversarial attacks on retrieval models
• Measuring Discrepancies in Airbnb Guest Acceptance Rates Using Anonymized Demographic Data