Coupled Multiple-response vs. Free-response Conceptual assessment: An Example from upper-division Physics

Abstract

Free-response research-based assessments, like the Colorado Upper-division Electrostatics Diagnostic (CUE), provide rich, fine-grained information about students' reasoning. However, because of the difficulties inherent in scoring these assessments, the majority of the large-scale conceptual assessments in physics are multiple-choice. To increase the scalability and usability of the CUE, we set out to create a new version of the assessment that preserves the insights afforded by a free-response format while exploiting the logistical advantages of a multiple-choice assessment. We used our extensive database of responses to the free-response CUE to construct distractors for a new version where students can select multiple responses and receive partial credit based on the accuracy and consistency of their selections. Here, we describe the development of this modified CUE format, which we call coupled multiple-response (CMR), and present data from direct comparisons of both versions. We find that the two formats have the same average score and perform similarly on multiple measures of validity and reliability, suggesting that the new version is a potentially viable alternative to the original CUE for the purpose of large-scale research-based assessment. We also compare the details of student responses on each of the two versions. While the CMR version does not capture the full scope of potential student responses, nearly three-quarters of our students' responses to the free-response version contained one or more elements that matched options provided on the CMR version.

28 pages, 6 figures, in Physical Review Special Topics - Physics Education Research

References in corpus (4)

• Surveying students' understanding of quantum mechanics in one spatial dimension
• Student understanding of symmetry and Gauss's law of electricity
• The Relativity Concept Inventory: development, analysis and results
• Can Free-Response Questions Be Approximated by Multiple-Choice Equivalents?

Cited by in corpus (11)

• Ready Student One: Exploring the predictors of student learning in virtual reality
• Development and Uses of Upper-division Conceptual Assessment
• Upper-division student difficulties with Separation of Variables
• Assessing Confidence in AI-Assisted Grading of Physics Exams through Psychometrics: An Exploratory Study
• Upper-division student difficulties with the Dirac delta function
• Assessing student reasoning in upper-division electricity and magnetism at Oregon State University
• Characterizing lab instructors' self-reported learning goals to inform development of an experimental modeling skills assessment
• Validation and analysis of the coupled multiple response Colorado upper-division electrostatics (CUE) diagnostic
• Student understanding of electric and magnetic fields in materials
• Assessing Learning Outcomes in Middle-Division Classical Mechanics: The Colorado Classical Mechanics/Math Methods Instrument
• Complex multiple-choice questions are inequitable for low-income and domestic students (and difficult for everyone)