Identifying Mislabeled Training Data

Abstract

This paper presents a new approach to identifying and eliminating mislabeled training instances for supervised learning. The goal of this approach is to improve classification accuracies produced by learning algorithms by improving the quality of the training data. Our approach uses a set of learning algorithms to create classifiers that serve as noise filters for the training data. We evaluate single algorithm, majority vote and consensus filters on five datasets that are prone to labeling errors. Our experiments illustrate that filtering significantly improves classification accuracy for noise levels up to 30 percent. An analytical and empirical evaluation of the precision of our approach shows that consensus filters are conservative at throwing away good data at the expense of retaining bad data and that majority filters are better at detecting bad data at the expense of throwing away good data. This suggests that for situations in which there is a paucity of data, consensus filters are preferable, whereas majority vote filters are preferable for situations with an abundance of data.

Cited by in corpus (16)

• Making Risk Minimization Tolerant to Label Noise
• Double Ramp Loss Based Reject Option Classifier
• Denoising Multi-Source Weak Supervision for Neural Text Classification
• Unsupervised classification to improve the quality of a bird song recording dataset
• Realistic On-The-Fly Outcomes of Planetary Collisions: Machine Learning Applied to Simulations of Giant Impacts
• OneLabeler: A Flexible System for Building Data Labeling Tools
• Identifying Mislabeled Instances in Classification Datasets
• Machine Learning with Confidential Computing: A Systematization of Knowledge
• Conditional anomaly detection with soft harmonic functions
• Beyond MeSH: Fine-Grained Semantic Indexing of Biomedical Literature based on Weak Supervision
• Robust and Efficient Boosting Method using the Conditional Risk
• Determining Sequence of Image Processing Technique (IPT) to Detect Adversarial Attacks
• "Is not the truth the truth?": Analyzing the Impact of User Validations for Bus In/Out Detection in Smartphone-based Surveys
• A Committee of Convolutional Neural Networks for Image Classication in the Concurrent Presence of Feature and Label Noise
• Ground-state properties via machine learning quantum constraints
• Ghost Loss to Question the Reliability of Training Data