3 papers
cs.LG2026
Pandora's Regret: A Proper Scoring Rule for Evaluating Sequential Search
Gerardo A. Flores, Yash Deshpande, Jannis R. Brea +1
In sequential search, alternatives are tested until the true class is found. Standard proper scoring rules like log loss are local, ignoring the ranking of competitors and misalign…
cs.LG2025
Aligning Evaluation with Clinical Priorities: Calibration, Label Shift, and Error Costs
Gerardo A. Flores, Alyssa H. Smith, Julia A. Fukuyama +1
Machine learning-based decision support systems are increasingly deployed in clinical settings, where probabilistic scoring functions are used to inform and prioritize patient mana…
cs.LG2025
A Consequentialist Critique of Binary Classification Evaluation: Theory, Practice, and Tools
Gerardo Flores, Abigail Schiff, Alyssa H. Smith +2
Machine learning-supported decisions, such as ordering diagnostic tests or determining preventive custody, often require converting probabilistic forecasts into binary classificati…