AI-aided Decision-Making in Education - Uncertainty, Explainability, and Human Responsibility
arXiv:2611.17858 · doi:10.1007/978-3-031-96855-6_2
Abstract
As Artificial Intelligence (AI)-based systems have become increasingly integrated into various aspects of everyday life, including education, understanding how these systems make decisions is of paramount importance. AI holds significant potential to enhance educational practices through the facilitation of personalized and adaptive learning, automation of assessment procedures, and assistance with administrative tasks. However, the opacity of AI-based decision-making in education (especially with deep learning and generative AI models) can hamper decision-making, especially when outcomes appear unjustified or incorrect. In this chapter, we argue that three key principles (uncertainty, explainability, and human responsibility) are essential for the legitimate use of AI in educational settings. Acknowledging the inherent uncertainties in AI-based decisions, analogous to those in human decision-making, is fundamental. Explainability is critical for ensuring that the functioning and outcomes of AI applications are transparent and comprehensible to both educators and learners. Furthermore, human responsibility is crucial, particularly in human decision-making, which remains indispensable across all stages of AI development and deployment. By examining these three principles, this chapter provides a comprehensive guide for stakeholders seeking to integrate AI thoughtfully and effectively into the educational system.
arXiv commentFabian Kieser and Paul Tschisgale contributed equally and share first authorship. This is a preprint of the following chapter: Kieser, F., Tschisgale, P., Wulff, P., "AI-Aided Decision-Making in Education: Uncertainty, Explainability, and Human Responsibility", published in "AI Roles and Responsibilities in Education", edited by S. Papadakis, 2025, Springer, reproduced with permission of Springer
References in corpus (11)
- Towards A Rigorous Science of Interpretable Machine Learning
- Semantics derived automatically from language corpora contain human-like biases
- A Review of Uncertainty Quantification in Deep Learning: Techniques, Applications and Challenges
- Aleatoric and Epistemic Uncertainty in Machine Learning: An Introduction to Concepts and Methods
- Explaining Explanations in AI
- Prediction-Based Decisions and Fairness: A Catalogue of Choices, Assumptions, and Definitions
- Physics task development of prospective physics teachers using ChatGPT
- Exploring Generative AI assisted feedback writing for students' written responses to a physics conceptual question with prompt engineering and few-shot learning
- Ethical and social risks of harm from Language Models
- Unreflected Acceptance -- Investigating the Negative Consequences of ChatGPT-Assisted Problem Solving in Physics Education
- Harnessing Large Language Models to Enhance Self-Regulated Learning via Formative Feedback