Who Decides if AI is Fair? The Labels Problem in Algorithmic Auditing

Abhilash Mishra, Yash Gorana

Labelled "ground truth" datasets are routinely used to evaluate and audit AI algorithms applied in high-stakes settings. However, there do not exist widely accepted benchmarks for…