Reliable isomorphic physics problem generation with large language models
arXiv:2607.13190
The paper describes an AI system that uses large language models to automatically create isomorphic physics problems, preserving the original problem structure while varying irrelevant details, and evaluates its usability for introductory mechanics.
Abstract
This study presents an AI-powered system for generating isomorphic physics problems using large language model (LLM)-based agent workflows. The system is designed around three practical goals: preserving the same conceptual and problem-solving structure as the original problems, varying construct-irrelevant features such as scenarios and numerical values, and producing questions that are directly usable without expert revision. The workflow combines prompt chaining, agent-based verification, and automated LaTeX compilation within a publicly accessible website hosted on a Raspberry Pi. To evaluate the system, we developed an eight-item rubric and tested the system using 13 multiple-choice questions from a calculus-based introductory Newtonian mechanics course. The evaluation results showed that 89% of the questions generated were rated as fully specified and directly usable. However, the system also showed limitations. The results suggest that LLM-based systems have significant potential for reliable instructional problem generation while also highlighting important challenges for future development.