Minority Class Oversampling for Tabular Data with Deep Generative Models

Abstract

In practice, machine learning experts are often confronted with imbalanced data. Without accounting for the imbalance, common classifiers perform poorly and standard evaluation metrics mislead the practitioners on the model's performance. A common method to treat imbalanced datasets is under- and oversampling. In this process, samples are either removed from the majority class or synthetic samples are added to the minority class. In this paper, we follow up on recent developments in deep learning. We take proposals of deep generative models, including our own, and study the ability of these approaches to provide realistic samples that improve performance on imbalanced classification tasks via oversampling. Across 160K+ experiments, we show that all of the new methods tend to perform better than simple baseline methods such as SMOTE, but require different under- and oversampling ratios to do so. Our experiments show that the way the method of sampling does not affect quality, but runtime varies widely. We also observe that the improvements in terms of performance metric, while shown to be significant when ranking the methods, often are minor in absolute terms, especially compared to the required effort. Furthermore, we notice that a large part of the improvement is due to undersampling, not oversampling. We make our code and testing framework available.

References in corpus (13)

• SMOTE: Synthetic Minority Over-sampling Technique
• Auto-Encoding Variational Bayes
• Conditional Generative Adversarial Nets
• Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks
• Categorical Reparameterization with Gumbel-Softmax
• Generating Multi-label Discrete Patient Records using Generative Adversarial Networks
• GANS for Sequences of Discrete Elements with the Gumbel-softmax Distribution
• Survey of resampling techniques for improving classification performance in unbalanced datasets
• Generating Multi-Categorical Samples with Generative Adversarial Networks
• Airline Passenger Name Record Generation using Generative Adversarial Networks
• BayesDB: A probabilistic programming system for querying the probable implications of data
• Improving Missing Data Imputation with Deep Generative Models
• HexaGAN: Generative Adversarial Nets for Real World Classification