Improving the Validity and Practical Usefulness of AI/ML Evaluations Using an Estimands Framework

Olivier Binette, Jerome P. Reiter

Commonly, AI or machine learning (ML) models are evaluated on benchmark datasets. This practice supports innovative methodological research, but benchmark performance can be poorly…