The Disparate Effects of Strategic Manipulation

Abstract

When consequential decisions are informed by algorithmic input, individuals may feel compelled to alter their behavior in order to gain a system's approval. Models of agent responsiveness, termed "strategic manipulation," analyze the interaction between a learner and agents in a world where all agents are equally able to manipulate their features in an attempt to "trick" a published classifier. In cases of real world classification, however, an agent's ability to adapt to an algorithm is not simply a function of her personal interest in receiving a positive classification, but is bound up in a complex web of social factors that affect her ability to pursue certain action responses. In this paper, we adapt models of strategic manipulation to capture dynamics that may arise in a setting of social inequality wherein candidate groups face different costs to manipulation. We find that whenever one group's costs are higher than the other's, the learner's equilibrium strategy exhibits an inequality-reinforcing phenomenon wherein the learner erroneously admits some members of the advantaged group, while erroneously excluding some members of the disadvantaged group. We also consider the effects of interventions in which a learner subsidizes members of the disadvantaged group, lowering their costs in order to improve her own classification performance. Here we encounter a paradoxical result: there exist cases in which providing a subsidy improves only the learner's utility while actually making both candidate groups worse-off--even the group receiving the subsidy. Our results reveal the potentially adverse social ramifications of deploying tools that attempt to evaluate an individual's "quality" when agents' capacities to adaptively respond differ.

arXiv comment29 pages, 4 figures

References in corpus (3)

• Avoiding Discrimination through Causal Reasoning
• Runaway Feedback Loops in Predictive Policing
• Bias In, Bias Out? Evaluating the Folk Wisdom

Cited by in corpus (45)

• Fairness in Machine Learning: A Survey
• Actionable Recourse in Linear Classification
• Improving Information from Manipulable Data
• A survey of algorithmic recourse: definitions, formulations, solutions, and prospects
• On Consequentialism and Fairness
• De-biasing "bias" measurement
• A Classification of Feedback Loops and Their Relation to Biases in Automated Decision-Making Systems
• What-is and How-to for Fairness in Machine Learning: A Survey, Reflection, and Perspective
• Optimal Decision Making Under Strategic Behavior
• Strategic Classification is Causal Modeling in Disguise
• Explainability for fair machine learning
• Outside the Echo Chamber: Optimizing the Performative Risk
• Equalizing Recourse across Groups
• Questioning the assumptions behind fairness solutions
• How Do Fair Decisions Fare in Long-term Qualification?
• Performative Prediction
• Decisions, Counterfactual Explanations and Strategic Behavior
• Allocating Opportunities in a Dynamic Model of Intergenerational Mobility
• Causal Modeling for Fairness in Dynamical Systems
• Maximizing Welfare with Incentive-Aware Evaluation Mechanisms
• Classification Under Misspecification: Halfspaces, Generalized Linear Models, and Connections to Evolvability
• Fair Machine Learning Under Partial Compliance
• The Role of Randomness and Noise in Strategic Classification
• Strategic Instrumental Variable Regression: Recovering Causal Relationships From Strategic Responses
• Unintended Selection: Persistent Qualification Rate Disparities and Interventions
• Causal Strategic Linear Regression
• Strategic Evaluation: Subjects, Evaluators, and Society
• PAC-Learning for Strategic Classification
• On the Current and Emerging Challenges of Developing Fair and Ethical AI Solutions in Financial Services
• Extracting Incentives from Black-Box Decisions
• From Predictions to Decisions: Using Lookahead Regularization
• Information Discrepancy in Strategic Learning
• Gaming Helps! Learning from Strategic Interactions in Natural Dynamics
• Dropping Standardized Testing for Admissions Trades Off Information and Access
• Who Leads and Who Follows in Strategic Classification?
• Measuring Fairness Under Unawareness of Sensitive Attributes: A Quantification-Based Approach
• Recourse for reclamation: Chatting with generative language models
• Test-optional Policies: Overcoming Strategic Behavior and Informational Gaps
• Linear Classifiers that Encourage Constructive Adaptation
• Best vs. All: Equity and Accuracy of Standardized Test Score Reporting
• Strategic Classification Made Practical
• The Limits of Optimal Pricing in the Dark
• Approximate Regions of Attraction in Learning with Decision-Dependent Distributions
• Strategic Classification in the Dark
• Stateful Strategic Regression