38 citations · 94 across the 9 of their papers we have counts for
10 papers
Anomaly Attribution with Likelihood Compensation
Tsuyoshi Idé, Amit Dhurandhar, Jiří Navrátil +2
This paper addresses the task of explaining anomalous predictions of a black-box regression model. When using a black-box model, such as one to predict building energy consumption…
Uncertainty Quantification 360: A Holistic Toolkit for Quantifying and Communicating the Uncertainty of AI
Soumya Ghosh, Q. Vera Liao, Karthikeyan Natesan Ramamurthy +4
In this paper, we describe an open source Python toolkit named Uncertainty Quantification 360 (UQ360) for the uncertainty quantification of AI models. The goal of this toolkit is t…
Uncertainty Characteristics Curves: A Systematic Assessment of Prediction Intervals
Jiri Navratil, Benjamin Elder, Matthew Arnold +2
Accurate quantification of model uncertainty has long been recognized as a fundamental requirement for trusted AI. In regression tasks, uncertainty is typically quantified using pr…
Learning Prediction Intervals for Model Performance
Benjamin Elder, Matthew Arnold, Anupama Murthi +1
Understanding model performance on unlabeled data is a fundamental challenge of developing, deploying, and maintaining AI systems. Model performance is typically evaluated using te…
Not Your Grandfathers Test Set: Reducing Labeling Effort for Testing
Begum Taskazan, Jiri Navratil, Matthew Arnold +3
Building and maintaining high-quality test sets remains a laborious and expensive task. As a result, test sets in the real world are often not properly kept up to date and drift fr…
Uncertainty Prediction for Deep Sequential Regression Using Meta Models
Jiri Navratil, Matthew Arnold, Benjamin Elder
Generating high quality uncertainty estimates for sequential regression, particularly deep recurrent networks, remains a challenging and open problem. Existing approaches often mak…
Towards Automating the AI Operations Lifecycle
Matthew Arnold, Jeffrey Boston, Michael Desmond +5
Today's AI deployments often require significant human involvement and skill in the operational stages of the model lifecycle, including pre-release testing, monitoring, problem di…
Accelerating Physics-Based Simulations Using Neural Network Proxies: An Application in Oil Reservoir Modeling
Jiri Navratil, Alan King, Jesus Rios +3
We develop a proxy model based on deep learning methods to accelerate the simulations of oil reservoirs--by three orders of magnitude--compared to industry-strength physics-based P…
Confidence Scoring Using Whitebox Meta-models with Linear Classifier Probes
Tongfei Chen, Jiří Navrátil, Vijay Iyengar +1
We propose a novel confidence scoring mechanism for deep neural networks based on a two-model paradigm involving a base model and a meta-model. The confidence score is learned by t…
Improved Answer Selection with Pre-Trained Word Embeddings
Rishav Chakravarti, Jiri Navratil, Cicero Nogueira dos Santos
This paper evaluates existing and newly proposed answer selection methods based on pre-trained word embeddings. Word embeddings are highly effective in various natural language pro…