The Boiling-Frog Problem of Physics Education
arXiv:2508.08842 · doi:10.1119/5.0296601
Abstract
It is astonishing how rapidly general-purpose AI has crossed familiar thresholds in introductory physics. Comparing outputs from successive models, GPT-5 Thinking moves far beyond the plug-and-chug tendencies seen earlier: on a classic elevator problem it works symbolically, notes when variables cancel, and verifies results; attempts to prompt novice-like behavior mainly affect tone, not method. On representation translation, the model scores 24/26 (92.3%) on TUG-Kv4.0. In a card-sorting proxy using two of my comprehensive finals (60 items), its categories reflect solution method rather than surface features. Solving those same exams, it attains 27/30 and 25/30, with most misses in ruler-based ray tracing and circuit interpretation. On epistemology, five independent CLASS runs yield 100\% favorable, indicating a simulated expert-like stance. Framed as a "boiling frog" problem, the paper argues for a decisive jump: retire credit-bearing unsupervised closed-response online assessments; grade process evidence; use paper, whiteboarding; shift weight to modeling, data, and authentic labs; require transparent, citable AI use; rebuild problem types; and lean on research-based instruction and peer learning. The opportunity is to foreground what AI cannot substitute for: modeling the world, arguing from evidence, and making principled approximations.
References in corpus (12)
- Could an Artificial-Intelligence agent pass an introductory physics course?
- How do physics students evaluate artificial intelligence responses on comprehension questions? A study on the perceived scientific accuracy and linguistic quality
- Performance of ChatGPT on the Test of Understanding Graphs in Kinematics
- Investigating the role of model-based reasoning while troubleshooting an electric circuit
- Physics Computational Literacy: An Exploratory Case Study Using Computational Essays
- On the Prevalence and Nature of Computational Instruction in Undergraduate Physics Programs across the United States
- Using Large Language Models to Assign Partial Credit to Students' Explanations of Problem-Solving Process: Grade at Human Level Accuracy with Grading Confidence Index and Personalized Student-facing Feedback
- Multilingual Performance of a Multimodal Artificial Intelligence System on Multisubject Physics Concept Inventories
- Assessing Confidence in AI-Assisted Grading of Physics Exams through Psychometrics: An Exploratory Study
- PhysPort use and growth: Supporting physics teaching with research-based resources since 2011
- Cheat sites and artificial intelligence usage in online introductory physics courses: what is the extent and what effect does it have on assessments?
- An empirical approach to interpreting card-sorting data