Value Cards: An Educational Toolkit for Teaching Social Impacts of Machine Learning through Deliberation

Abstract

Recently, there have been increasing calls for computer science curricula to complement existing technical training with topics related to Fairness, Accountability, Transparency, and Ethics. In this paper, we present Value Card, an educational toolkit to inform students and practitioners of the social impacts of different machine learning models via deliberation. This paper presents an early use of our approach in a college-level computer science course. Through an in-class activity, we report empirical data for the initial effectiveness of our approach. Our results suggest that the use of the Value Cards toolkit can improve students' understanding of both the technical definitions and trade-offs of performance metrics and apply them in real-world contexts, help them recognize the significance of considering diverse social values in the development of deployment of algorithmic systems, and enable them to communicate, negotiate and synthesize the perspectives of diverse stakeholders. Our study also demonstrates a number of caveats we need to consider when using the different variants of the Value Cards toolkit. Finally, we discuss the challenges as well as future applications of our approach.

References in corpus (3)

• Improving fairness in machine learning systems: What do industry practitioners need?
• Fairness Testing: Testing Software for Discrimination
• Lessons from Archives: Strategies for Collecting Sociocultural Data in Machine Learning

Cited by in corpus (12)

• Accountability in an Algorithmic Society: Relationality, Responsibility, and Robustness in Machine Learning
• Seeing Like a Toolkit: How Toolkits Envision the Work of AI Ethics
• Exploring How Machine Learning Practitioners (Try To) Use Fairness Toolkits
• Deliberating with AI: Improving Decision-Making for the Future through Participatory AI Design and Stakeholder Deliberation
• Farsight: Fostering Responsible AI Awareness During AI Application Prototyping
• Robots Enact Malignant Stereotypes
• "That's important, but...": How Computer Science Researchers Anticipate Unintended Consequences of Their Research Innovations
• An Uncommon Task: Participatory Design in Legal AI
• REAL ML: Recognizing, Exploring, and Articulating Limitations of Machine Learning Research
• Saliency Cards: A Framework to Characterize and Compare Saliency Methods
• Supporting Industry Computing Researchers in Assessing, Articulating, and Addressing the Potential Negative Societal Impact of Their Work
• CARDinality: Interactive Card-shaped Robots with Locomotion and Haptics using Vibration