The Algorithmic Imprint

Abstract

When algorithmic harms emerge, a reasonable response is to stop using the algorithm to resolve concerns related to fairness, accountability, transparency, and ethics (FATE). However, just because an algorithm is removed does not imply its FATE-related issues cease to exist. In this paper, we introduce the notion of the "algorithmic imprint" to illustrate how merely removing an algorithm does not necessarily undo or mitigate its consequences. We operationalize this concept and its implications through the 2020 events surrounding the algorithmic grading of the General Certificate of Education (GCE) Advanced (A) Level exams, an internationally recognized UK-based high school diploma exam administered in over 160 countries. While the algorithmic standardization was ultimately removed due to global protests, we show how the removal failed to undo the algorithmic imprint on the sociotechnical infrastructures that shape students', teachers', and parents' lives. These events provide a rare chance to analyze the state of the world both with and without algorithmic mediation. We situate our case study in Bangladesh to illustrate how algorithms made in the Global North disproportionately impact stakeholders in the Global South. Chronicling more than a year-long community engagement consisting of 47 inter-views, we present the first coherent timeline of "what" happened in Bangladesh, contextualizing "why" and "how" they happened through the lenses of the algorithmic imprint and situated algorithmic fairness. Analyzing these events, we highlight how the contours of the algorithmic imprints can be inferred at the infrastructural, social, and individual levels. We share conceptual and practical implications around how imprint-awareness can (a) broaden the boundaries of how we think about algorithmic impact, (b) inform how we design algorithms, and (c) guide us in AI governance.

Accepted to ACM FAccT 2022

References in corpus (6)

• 'It's Reducing a Human Being to a Percentage'; Perceptions of Justice in Algorithmic Decisions
• Expanding Explainability: Towards Social Transparency in AI systems
• Closing the AI Accountability Gap: Defining an End-to-End Framework for Internal Algorithmic Auditing
• Everyday algorithm auditing: Understanding the power of everyday users in surfacing harmful algorithmic behaviors
• Black Feminist Musings on Algorithmic Oppression
• Automated Rationale Generation: A Technique for Explainable AI and its Effects on Human Perceptions

Cited by in corpus (5)

• Harms from Increasingly Agentic Algorithmic Systems
• The Fall of an Algorithm: Characterizing the Dynamics Toward Abandonment
• Materiality and Risk in the Age of Pervasive AI Sensors
• How Formerly Incarcerated People Envision Technologies for Prison Parole
• "Till I can get my satisfaction": Open Questions in the Public Desire to Punish AI