Dice in the Black Box: User Experiences with an Inscrutable Algorithm

Abstract

We demonstrate that users may be prone to place an inordinate amount of trust in black box algorithms that are framed as intelligent. We deploy an algorithm that purportedly assesses the positivity and negativity of a users' writing emotional writing. In actuality, the algorithm responds in a random fashion. We qualitatively examine the paths to trust that users followed while testing the system. In light of the ease with which users may trust systems exhibiting "intelligent behavior" we recommend corrective approaches.

Cited by in corpus (5)

• The Authority of "Fair" in Machine Learning
• Explanation Strategies as an Empirical-Analytical Lens for Socio-Technical Contextualization of Machine Learning Interpretability
• Progressive Disclosure: Designing for Effective Transparency
• What Are You Hiding? Algorithmic Transparency and User Perceptions
• "I had a solid theory before but it's falling apart": Polarizing Effects of Algorithmic Transparency