Social Biases in Automatic Evaluation Metrics for NLG

Mingqi Gao, Xiaojun Wan

Many studies have revealed that word embeddings, language models, and models for specific downstream tasks in NLP are prone to social biases, especially gender bias. Recently these…