Null Compliance: NYC Local Law 144 and the Challenges of Algorithm Accountability

Abstract

In July 2023, New York City became the first jurisdiction globally to mandate bias audits for commercial algorithmic systems, specifically for automated employment decisions systems (AEDTs) used in hiring and promotion. Local Law 144 (LL 144) requires AEDTs to be independently audited annually for race and gender bias, and the audit report must be publicly posted. Additionally, employers are obligated to post a transparency notice with the job listing. In this study, 155 student investigators recorded 391 employers' compliance with LL 144 and the user experience for prospective job applicants. Among these employers, 18 posted audit reports and 13 posted transparency notices. These rates could potentially be explained by a significant limitation in the accountability mechanisms enacted by LL 144. Since the law grants employers substantial discretion over whether their system is in scope of the law, a null result cannot be said to indicate non-compliance, a condition we call ``null compliance." Employer discretion may also explain our finding that nearly all audits reported an impact factor over 0.8, a rule of thumb often used in employment discrimination cases. We also find that the benefit of LL 144 to ordinary job seekers is limited due to shortcomings in accessibility and usability. Our findings offer important lessons for policy-makers as they consider regulating algorithmic systems, particularly the degree of discretion to grant to regulated parties and the limitations of relying on transparency and end-user accountability.

References in corpus (5)

• Mitigating Bias in Algorithmic Hiring: Evaluating Claims and Practices
• Dark Patterns at Scale: Findings from a Crawl of 11K Shopping Websites
• The Fallacy of AI Functionality
• Beyond Dark Patterns: A Concept-Based Framework for Ethical Software Design
• The four-fifths rule is not disparate impact: a woeful tale of epistemic trespassing in algorithmic fairness

Cited by in corpus (7)

• The Silicon Ceiling: Auditing GPT's Race and Gender Biases in Hiring
• Towards AI Accountability Infrastructure: Gaps and Opportunities in AI Audit Tooling
• Regulating Algorithmic Management: A Multi-Stakeholder Study of Challenges in Aligning Software and the Law for Workplace Scheduling
• Navigating Automated Hiring: Perceptions, Strategy Use, and Outcomes Among Young Job Seekers
• Compliant But Unsatisfactory: The Gap Between Auditing Standards and Practices for Probabilistic Genotyping Software
• The Fair Game: Auditing & Debiasing AI Algorithms Over Time
• Fairness is in the details: Face Dataset Auditing