Value Cards: An Educational Toolkit for Teaching Social Impacts of Machine Learning through Deliberation
arXiv:2010.11411 · doi:10.1145/3442188.3445971
Abstract
Recently, there have been increasing calls for computer science curricula to complement existing technical training with topics related to Fairness, Accountability, Transparency, and Ethics. In this paper, we present Value Card, an educational toolkit to inform students and practitioners of the social impacts of different machine learning models via deliberation. This paper presents an early use of our approach in a college-level computer science course. Through an in-class activity, we report empirical data for the initial effectiveness of our approach. Our results suggest that the use of the Value Cards toolkit can improve students' understanding of both the technical definitions and trade-offs of performance metrics and apply them in real-world contexts, help them recognize the significance of considering diverse social values in the development of deployment of algorithmic systems, and enable them to communicate, negotiate and synthesize the perspectives of diverse stakeholders. Our study also demonstrates a number of caveats we need to consider when using the different variants of the Value Cards toolkit. Finally, we discuss the challenges as well as future applications of our approach.
References in corpus (9)
- Model Cards for Model Reporting
- Improving fairness in machine learning systems: What do industry practitioners need?
- The What-If Tool: Interactive Probing of Machine Learning Models
- Fairness and Accountability Design Needs for Algorithmic Support in High-Stakes Public Sector Decision-Making
- Fairness Testing: Testing Software for Discrimination
- Preventing Fairness Gerrymandering: Auditing and Learning for Subgroup Fairness
- Lessons from Archives: Strategies for Collecting Sociocultural Data in Machine Learning
- Why Is My Classifier Discriminatory?
- A Reductions Approach to Fair Classification
Cited by in corpus (12)
- Accountability in an Algorithmic Society: Relationality, Responsibility, and Robustness in Machine Learning
- Seeing Like a Toolkit: How Toolkits Envision the Work of AI Ethics
- Exploring How Machine Learning Practitioners (Try To) Use Fairness Toolkits
- Deliberating with AI: Improving Decision-Making for the Future through Participatory AI Design and Stakeholder Deliberation
- Farsight: Fostering Responsible AI Awareness During AI Application Prototyping
- Robots Enact Malignant Stereotypes
- "That's important, but...": How Computer Science Researchers Anticipate Unintended Consequences of Their Research Innovations
- An Uncommon Task: Participatory Design in Legal AI
- REAL ML: Recognizing, Exploring, and Articulating Limitations of Machine Learning Research
- Saliency Cards: A Framework to Characterize and Compare Saliency Methods
- Supporting Industry Computing Researchers in Assessing, Articulating, and Addressing the Potential Negative Societal Impact of Their Work
- CARDinality: Interactive Card-shaped Robots with Locomotion and Haptics using Vibration