HardVis: Visual Analytics to Handle Instance Hardness Using Undersampling and Oversampling Techniques

Abstract

Despite the tremendous advances in machine learning (ML), training with imbalanced data still poses challenges in many real-world applications. Among a series of diverse techniques to solve this problem, sampling algorithms are regarded as an efficient solution. However, the problem is more fundamental, with many works emphasizing the importance of instance hardness. This issue refers to the significance of managing unsafe or potentially noisy instances that are more likely to be misclassified and serve as the root cause of poor classification performance. This paper introduces HardVis, a visual analytics system designed to handle instance hardness mainly in imbalanced classification scenarios. Our proposed system assists users in visually comparing different distributions of data types, selecting types of instances based on local characteristics that will later be affected by the active sampling method, and validating which suggestions from undersampling or oversampling techniques are beneficial for the ML model. Additionally, rather than uniformly undersampling/oversampling a specific class, we allow users to find and sample easy and difficult to classify training instances from all classes. Users can explore subsets of data from different perspectives to decide all those parameters, while HardVis keeps track of their steps and evaluates the model's predictive performance in a test set separately. The end result is a well-balanced data set that boosts the predictive power of the ML model. The efficacy and effectiveness of HardVis are demonstrated with a hypothetical usage scenario and a use case. Finally, we also look at how useful our system is based on feedback we received from ML experts.

This manuscript is accepted for publication in Computer Graphics Forum (CGF)

References in corpus (10)

• XGBoost: A Scalable Tree Boosting System
• SMOTE: Synthetic Minority Over-sampling Technique
• The State of the Art in Enhancing Trust in Machine Learning Models with the Use of Visualizations
• t-viSNE: Interactive Assessment and Interpretation of t-SNE Projections
• StackGenVis: Alignment of Data, Algorithms, and Models for Stacking Ensemble Learning Using Performance Metrics
• Explaining Vulnerabilities to Adversarial Machine Learning through Visual Analytics
• FeatureEnVi: Visual Analytics for Feature Engineering Using Stepwise Selection and Semi-Automatic Extraction Approaches
• VisEvol: Visual Analytics to Support Hyperparameter Search through Evolutionary Optimization
• Classifier-Guided Visual Correction of Noisy Labels for Image Classification Tasks
• REsCUE: A framework for REal-time feedback on behavioral CUEs using multimodal anomaly detection

Cited by in corpus (2)

• Visual Analytics for Explainable and Trustworthy Artificial Intelligence
• DeforestVis: Behavior Analysis of Machine Learning Models with Surrogate Decision Stumps