Endogenous Macrodynamics in Algorithmic Recourse
arXiv:2308.08187 · doi:10.1109/SaTML54575.2023.00036
Abstract
Existing work on Counterfactual Explanations (CE) and Algorithmic Recourse (AR) has largely focused on single individuals in a static environment: given some estimated model, the goal is to find valid counterfactuals for an individual instance that fulfill various desiderata. The ability of such counterfactuals to handle dynamics like data and model drift remains a largely unexplored research challenge. There has also been surprisingly little work on the related question of how the actual implementation of recourse by one individual may affect other individuals. Through this work, we aim to close that gap. We first show that many of the existing methodologies can be collectively described by a generalized framework. We then argue that the existing framework does not account for a hidden external cost of recourse, that only reveals itself when studying the endogenous dynamics of recourse at the group level. Through simulation experiments involving various state-of the-art counterfactual generators and several benchmark datasets, we generate large numbers of counterfactuals and study the resulting domain and model shifts. We find that the induced shifts are substantial enough to likely impede the applicability of Algorithmic Recourse in some situations. Fortunately, we find various strategies to mitigate these concerns. Our simulation framework for studying recourse dynamics is fast and opensourced.
arXiv comment12 pages, 11 figures. Originally published at the 2023 IEEE Conference on Secure and Trustworthy Machine Learning (SaTML). IEEE holds the copyright
References in corpus (12)
- Scikit-learn: Machine Learning in Python
- Explaining Machine Learning Classifiers through Diverse Counterfactual Explanations
- Deep Neural Networks and Tabular Data: A Survey
- Actionable Recourse in Linear Classification
- Failing Loudly: An Empirical Study of Methods for Detecting Dataset Shift
- Fashionable Modelling with Flux
- Towards Realistic Individual Recourse and Actionable Explanations in Black-Box Decision Making Systems
- Algorithmic Recourse: from Counterfactual Explanations to Interventions
- Getting a CLUE: A Method for Explaining Uncertainty Estimates
- Towards Robust and Reliable Algorithmic Recourse
- A Framework and Benchmarking Study for Counterfactual Generating Methods on Tabular Data
- Generating collective counterfactual explanations in score-based classification via mathematical optimization