Can an AI-tool grade assignments in an introductory physics course?

Abstract

Problem solving is an integral part of any physics curriculum, and most physics instructors would likely agree that the associated learner competencies are best assessed by considering the solution path: not only the final solution matters, but also how the learner arrived there. Unfortunately, providing meaningful feedback on written derivations is much more labor and resource intensive than only grading the outcome: currently, the latter can be done by computer, while the former involves handwritten solutions that need to be graded by humans. This exploratory study proposes an AI-assisted workflow for grading written physics-problem solutions, and it evaluates the viability of the actual grading step using GPT-4. It is found that the AI-tool is capable of providing feedback that can be helpful in formative assessment scenarios, but that for summative scenarios, particularly those that are high-stakes, it should only be used for an initial round of grading that sorts and flags solution approaches.

References in corpus (3)

• Showing Academic Performance Predictions during Term Planning: Effects on Students' Decisions, Behaviors, and Preferences
• Could an Artificial-Intelligence agent pass an introductory physics course?
• How do physics students evaluate artificial intelligence responses on comprehension questions? A study on the perceived scientific accuracy and linguistic quality

Cited by in corpus (11)

• Performance of ChatGPT on the Test of Understanding Graphs in Kinematics
• ChatGPT as a tool for honing teachers' Socratic dialogue skills
• Using Large Language Models to Assign Partial Credit to Students' Explanations of Problem-Solving Process: Grade at Human Level Accuracy with Grading Confidence Index and Personalized Student-facing Feedback
• Assessing Confidence in AI-Assisted Grading of Physics Exams through Psychometrics: An Exploratory Study
• Ethel: A Virtual Teaching Assistant
• Evaluating the capability of large language models to personalize science texts for diverse middle-school-age learners
• Using AI Large Language Models for Grading in Education: A Hands-On Test for Physics
• Opportunities and Challenges in Harnessing Digital Technology for Effective Teaching and Learning
• Exploring Large Language Models (LLMs) through interactive Python activities
• Can Large Language Models Correctly Interpret Equations with Errors?
• Superstudent intelligence in thermodynamics