3 papers
cs.LG2026
Pandora's Regret: A Proper Scoring Rule for Evaluating Sequential Search
Gerardo A. Flores, Yash Deshpande, Jannis R. Brea +1
In sequential search, alternatives are tested until the true class is found. Standard proper scoring rules like log loss are local, ignoring the ranking of competitors and misalign…
cs.LG2026
A Consequentialist Critique of Binary Classification Evaluation: Theory, Practice, and Tools
Gerardo Flores, Abigail Schiff, Alyssa H. Smith +2
Machine learning-supported decisions, such as ordering diagnostic tests or determining preventive custody, often require converting probabilistic forecasts into binary classificati…
cs.LG2025
Aligning Evaluation with Clinical Priorities: Calibration, Label Shift, and Error Costs
Gerardo A. Flores, Alyssa H. Smith, Julia A. Fukuyama +1
Machine learning-based decision support systems are increasingly deployed in clinical settings, where probabilistic scoring functions are used to inform and prioritize patient mana…