SparCAssist: A Model Risk Assessment Assistant Based on Sparse Generated Counterfactuals

Abstract

We introduce SparcAssist, a general-purpose risk assessment tool for the machine learning models trained for language tasks. It evaluates models' risk by inspecting their behavior on counterfactuals, namely out-of-distribution instances generated based on the given data instance. The counterfactuals are generated by replacing tokens in rational subsequences identified by ExPred, while the replacements are retrieved using HotFlip or Masked-Language-Model-based algorithms. The main purpose of our system is to help the human annotators to assess the model's risk on deployment. The counterfactual instances generated during the assessment are the by-product and can be used to train more robust NLP models in the future.

arXiv comment5 pages, 2 figures, submitted and accepted as a demo paper in SIGIR2022

References in corpus (9)

• Axiomatic Attribution for Deep Networks
• Learning the Difference that Makes a Difference with Counterfactually-Augmented Data
• Explain and Predict, and then Predict Again
• Posthoc Interpretability of Learning to Rank Models using Secondary Training Data
• When is invariance useful in an Out-of-Distribution Generalization problem ?
• Tempas: Temporal Archive Search Based on Tags
• FaxPlainAC: A Fact-Checking Tool Based on EXPLAINable Models with HumAn Correction in the Loop
• Discovering Entities with Just a Little Help from You
• The Irrationality of Neural Rationale Models