Data Cleansing for Models Trained with SGD

Abstract

Data cleansing is a typical approach used to improve the accuracy of machine learning models, which, however, requires extensive domain knowledge to identify the influential instances that affect the models. In this paper, we propose an algorithm that can suggest influential instances without using any domain knowledge. With the proposed method, users only need to inspect the instances suggested by the algorithm, implying that users do not need extensive knowledge for this procedure, which enables even non-experts to conduct data cleansing and improve the model. The existing methods require the loss function to be convex and an optimal model to be obtained, which is not always the case in modern machine learning. To overcome these limitations, we propose a novel approach specifically designed for the models trained with stochastic gradient descent (SGD). The proposed method infers the influential instances by retracing the steps of the SGD while incorporating intermediate models computed in each step. Through experiments, we demonstrate that the proposed method can accurately infer the influential instances. Moreover, we used MNIST and CIFAR10 to show that the models can be effectively improved by removing the influential instances suggested by the proposed method.

References in corpus (3)

• Understanding Black-box Predictions via Influence Functions
• Interpreting Black Box Predictions using Fisher Kernels
• Training Set Debugging Using Trusted Items

Cited by in corpus (7)

• HYDRA: Hypergradient Data Relevance Analysis for Interpreting Deep Neural Networks
• Understanding Instance-based Interpretability of Variational Auto-Encoders
• The Thousand Faces of Explainable AI Along the Machine Learning Life Cycle: Industrial Reality and Current State of Research
• Neural Network Libraries: A Deep Learning Framework Designed from Engineers' Perspectives
• Influence Estimation for Generative Adversarial Networks
• Data Cleansing for Deep Neural Networks with Storage-efficient Approximation of Influence Functions
• Online Robust and Adaptive Learning from Data Streams