Could an Artificial-Intelligence agent pass an introductory physics course?
arXiv:2301.12127 · doi:10.1103/PhysRevPhysEducRes.19.010132
Abstract
Massive pre-trained language models have garnered attention and controversy due to their ability to generate human-like responses: attention due to their frequent indistinguishability from human-generated phraseology and narratives, and controversy due to the fact that their convincingly presented arguments and facts are frequently simply false. Just how human-like are these responses when it comes to dialogues about physics, in particular about the standard content of introductory physics courses? This study explores that question by having ChatGTP, the pre-eminent language model in 2023, work through representative assessment content of an actual calculus-based physics course and grading the responses in the same way human responses would be graded. As it turns out, ChatGPT would narrowly pass this course while exhibiting many of the preconceptions and errors of a beginning learner.
References in corpus (5)
- Advanced tools for smartphone-based experiments: phyphox
- Reinventing College Physics for Biologists: Explicating an epistemological curriculum
- Cinema, Fermi Problems, & General Education
- Students' perspectives on computational challenges in physics class
- Understanding Student Computational Thinking with Computational Modeling
Cited by in corpus (22)
- Summary of ChatGPT-Related Research and Perspective Towards the Future of Large Language Models
- How understanding large language models can inform the use of ChatGPT in physics education
- Exploring Generative AI assisted feedback writing for students' written responses to a physics conceptual question with prompt engineering and few-shot learning
- How do physics students evaluate artificial intelligence responses on comprehension questions? A study on the perceived scientific accuracy and linguistic quality
- Performance of ChatGPT on the Test of Understanding Graphs in Kinematics
- Can an AI-tool grade assignments in an introductory physics course?
- Empirical assessment of ChatGPT's answering capabilities in natural science and engineering
- A comparison of Human, GPT-3.5, and GPT-4 Performance in a University-Level Coding Course
- ChatGPT as a tool for honing teachers' Socratic dialogue skills
- Multilingual Performance of a Multimodal Artificial Intelligence System on Multisubject Physics Concept Inventories
- Assessing Confidence in AI-Assisted Grading of Physics Exams through Psychometrics: An Exploratory Study
- Performance of ChatGPT on tasks involving physics visual representations: the case of the Brief Electricity and Magnetism Assessment
- Cheat sites and artificial intelligence usage in online introductory physics courses: what is the extent and what effect does it have on assessments?
- Ethel: A Virtual Teaching Assistant
- Evaluating GPT- and Reasoning-based Large Language Models on Physics Olympiad Problems: Surpassing Human Performance and Implications for Educational Assessment
- Using artificial-intelligence tools to make LaTeX content accessible to blind readers
- Can ChatGPT pass a physics degree? Making a case for reformation of assessment of undergraduate degrees
- Opportunities and Challenges in Harnessing Digital Technology for Effective Teaching and Learning
- Innovative approaches to high school physics competitions: Harnessing the power of AI and open science
- The Boiling-Frog Problem of Physics Education
- AI-supported data analysis boosts student motivation and reduces stress in physics education
- When AI Evaluates Its Own Work: Validating Learner-Initiated, AI-Generated Physics Practice Problems