Investigating Group Distributionally Robust Optimization for Deep Imbalanced Learning: A Case Study of Binary Tabular Data Classification

Abstract

One of the most studied machine learning challenges that recent studies have shown the susceptibility of deep neural networks to is the class imbalance problem. While concerted research efforts in this direction have been notable in recent years, findings have shown that the canonical learning objective, empirical risk minimization (ERM), is unable to achieve optimal imbalance learning in deep neural networks given its bias to the majority class. An alternative learning objective, group distributionally robust optimization (gDRO), is investigated in this study for imbalance learning, focusing on tabular imbalanced data as against image data that has dominated deep imbalance learning research. Contrary to minimizing average per instance loss as in ERM, gDRO seeks to minimize the worst group loss over the training data. Experimental findings in comparison with ERM and classical imbalance methods using four popularly used evaluation metrics in imbalance learning across several benchmark imbalance binary tabular data of varying imbalance ratios reveal impressive performance of gDRO, outperforming other compared methods in terms of g-mean and roc-auc.

arXiv comment10 pages

References in corpus (11)

• PyTorch: An Imperative Style, High-Performance Deep Learning Library
• mixup: Beyond Empirical Risk Minimization
• A systematic study of the class imbalance problem in convolutional neural networks
• Deep Neural Networks and Tabular Data: A Survey
• Distributionally Robust Neural Networks for Group Shifts: On the Importance of Regularization for Worst-Case Generalization
• Balanced Meta-Softmax for Long-Tailed Visual Recognition
• What is the Effect of Importance Weighting in Deep Learning?
• Remix: Rebalanced Mixup
• Identifying and Compensating for Feature Deviation in Imbalanced Deep Learning
• Scalable Twin Neural Networks for Classification of Unbalanced Data
• A Cost-Sensitive Deep Belief Network for Imbalanced Classification