A comparison of Hake's g and Cohen's d for analyzing gains on concept inventories

Abstract

Measuring student learning is a complicated but necessary task for understanding the effectiveness of instruction and issues of equity in college STEM courses. Our investigation focused on the implications on claims about student learning that result from choosing between one of two commonly used methods for analyzing shifts in concept inventories. The methods are: Hake's gain (g), which is the most common method used in physics education research and other discipline based education research fields, and Cohen's d, which is broadly used in education research and many other fields. Data for the analyses came from the Learning Assistant Supported Student Outcomes (LASSO) database and included test scores from 4,551 students on physics, chemistry, biology, and math concept inventories from 89 courses at 17 institutions from across the United States. We compared the two methods across all of the concept inventories. The results showed that the two methods led to different inferences about student learning and equity due to g being biased in favor of high pretest populations. Recommendations for the analysis and reporting of findings on student learning data are included.

References in corpus (2)

• Secondary Analysis of Teaching Methods in Introductory Physics: a 50k-Student Study
• Beyond performance metrics: Examining a decrease in students' physics self-efficacy through a social network lens

Cited by in corpus (11)

• Effectiveness of flip teaching on engineering students' performance in the physics lab
• Modernizing use of regression models in physics education research: a review of hierarchical linear modeling
• Investigating and improving student understanding of the basics of quantum computing
• Quantitatively ranking incorrect responses to multiple-choice questions using item response theory
• Authentic Science Experiences with STEM Datasets: Post-secondary Results and Potential Gender Influences
• Positive attitudinal shifts and a narrowing gender gap: Do expertlike attitudes correlate to higher learning gains for women in the physics classroom?
• Bias in physics peer recognition does not explain gaps in perceived recognition
• A three-year comparative study of dominant misconceptions among first-year physics students at a South African university
• Improving student understanding of quantum measurement in infinite-dimensional Hilbert space using a research-based multiple-choice question sequence
• Novel computational approaches for ratio distributions with an application to Hake's ratio in effect size measurement
• Instructor Framing and Incentives Shape Physics Students' Engagement and Learning Gains from an Inquiry-Based Electrostatics Tutorial on the Method of Images