Are We Grading Properly? Understanding Failure Modes in Medical Benchmarks

Prithvi Dixit, Pedram Hosseini

Medical evaluation is shifting from static option-based questioning to realistic clinical scenarios with open-ended output modes. Grading these at scale naively, however, is expens…