3 citations · 4 across the 2 of their papers we have counts for
4 papers · 1 filter
Uncertainty Characteristics Curves: A Systematic Assessment of Prediction Intervals
Jiri Navratil, Benjamin Elder, Matthew Arnold +2
Accurate quantification of model uncertainty has long been recognized as a fundamental requirement for trusted AI. In regression tasks, uncertainty is typically quantified using pr…
Learning Prediction Intervals for Model Performance
Benjamin Elder, Matthew Arnold, Anupama Murthi +1
Understanding model performance on unlabeled data is a fundamental challenge of developing, deploying, and maintaining AI systems. Model performance is typically evaluated using te…
Not Your Grandfathers Test Set: Reducing Labeling Effort for Testing
Begum Taskazan, Jiri Navratil, Matthew Arnold +3
Building and maintaining high-quality test sets remains a laborious and expensive task. As a result, test sets in the real world are often not properly kept up to date and drift fr…
Towards Automating the AI Operations Lifecycle
Matthew Arnold, Jeffrey Boston, Michael Desmond +5
Today's AI deployments often require significant human involvement and skill in the operational stages of the model lifecycle, including pre-release testing, monitoring, problem di…