"A cold, technical decision-maker": Can AI provide explainability, negotiability, and humanity?
arXiv:2012.00874
Abstract
Algorithmic systems are increasingly deployed to make decisions in many areas of people's lives. The shift from human to algorithmic decision-making has been accompanied by concern about potentially opaque decisions that are not aligned with social values, as well as proposed remedies such as explainability. We present results of a qualitative study of algorithmic decision-making, comprised of five workshops conducted with a total of 60 participants in Finland, Germany, the United Kingdom, and the United States. We invited participants to reason about decision-making qualities such as explainability and accuracy in a variety of domains. Participants viewed AI as a decision-maker that follows rigid criteria and performs mechanical tasks well, but is largely incapable of subjective or morally complex judgments. We discuss participants' consideration of humanity in decision-making, and introduce the concept of 'negotiability,' the ability to go beyond formal criteria and work flexibly around the system.
arXiv comment23 pages, 1 appendix, 4 tables
References in corpus (6)
- Towards A Rigorous Science of Interpretable Machine Learning
- European Union regulations on algorithmic decision-making and a "right to explanation"
- Actionable Recourse in Linear Classification
- A systematic review and taxonomy of explanations in decision support and recommender systems
- The Promise and Peril of Human Evaluation for Model Interpretability
- Long-Term Trends in the Public Perception of Artificial Intelligence