Does Gender Matter? Towards Fairness in Dialogue Systems

Abstract

Recently there are increasing concerns about the fairness of Artificial Intelligence (AI) in real-world applications such as computer vision and recommendations. For example, recognition algorithms in computer vision are unfair to black people such as poorly detecting their faces and inappropriately identifying them as "gorillas". As one crucial application of AI, dialogue systems have been extensively applied in our society. They are usually built with real human conversational data; thus they could inherit some fairness issues which are held in the real world. However, the fairness of dialogue systems has not been well investigated. In this paper, we perform a pioneering study about the fairness issues in dialogue systems. In particular, we construct a benchmark dataset and propose quantitative measures to understand fairness in dialogue models. Our studies demonstrate that popular dialogue models show significant prejudice towards different genders and races. Besides, to mitigate the bias in dialogue systems, we propose two simple but effective debiasing methods. Experiments show that our methods can reduce the bias in dialogue systems significantly. The dataset and the implementation are released to foster fairness research in dialogue systems.

arXiv commentAccepted by COLING 2020

References in corpus (11)

• Sequence to Sequence Learning with Neural Networks
• Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddings
• Fairness Constraints: Mechanisms for Fair Classification
• A Convex Framework for Fair Regression
• Compositional Fairness Constraints for Graph Embeddings
• Scalable Fair Clustering
• Proportionally Fair Clustering
• Complex Sequential Question Answering: Towards Learning to Converse Over Linked Question Answer Pairs with a Knowledge Graph
• Say What I Want: Towards the Dark Side of Neural Dialogue Models
• Chat as Expected: Learning to Manipulate Black-box Neural Dialogue Models
• Assessing Gender Bias in Machine Translation -- A Case Study with Google Translate

Cited by in corpus (3)

• Anticipating Safety Issues in E2E Conversational AI: Framework and Tooling
• Jointly Attacking Graph Neural Network and its Explanations
• MABR: Multilayer Adversarial Bias Removal Without Prior Bias Knowledge