Measuring Fairness with Biased Rulers: A Survey on Quantifying Biases in Pretrained Language Models

Pieter Delobelle, Ewoenam Kwaku Tokpo, Toon Calders +1

An increasing awareness of biased patterns in natural language processing resources, like BERT, has motivated many metrics to quantify `bias' and `fairness'. But comparing the resu…