Delving into Deep Imbalanced Regression

Abstract

Real-world data often exhibit imbalanced distributions, where certain target values have significantly fewer observations. Existing techniques for dealing with imbalanced data focus on targets with categorical indices, i.e., different classes. However, many tasks involve continuous targets, where hard boundaries between classes do not exist. We define Deep Imbalanced Regression (DIR) as learning from such imbalanced data with continuous targets, dealing with potential missing data for certain target values, and generalizing to the entire target range. Motivated by the intrinsic difference between categorical and continuous label space, we propose distribution smoothing for both labels and features, which explicitly acknowledges the effects of nearby targets, and calibrates both label and learned feature distributions. We curate and benchmark large-scale DIR datasets from common real-world tasks in computer vision, natural language processing, and healthcare domains. Extensive experiments verify the superior performance of our strategies. Our work fills the gap in benchmarks and techniques for practical imbalanced regression problems. Code and data are available at https://github.com/YyzHarry/imbalanced-regression.

ICML 2021 (Long Oral)

References in corpus (7)

• Adam: A Method for Stochastic Optimization
• SMOTE: Synthetic Minority Over-sampling Technique
• A systematic study of the class imbalance problem in convolutional neural networks
• Depth Map Prediction from a Single Image using a Multi-Scale Deep Network
• AdaBins: Depth Estimation using Adaptive Bins
• Manifold Mixup: Better Representations by Interpolating Hidden States
• Large-Scale Long-Tailed Recognition in an Open World

Cited by in corpus (9)

• Rebalanced Zero-shot Learning
• The Portiloop: a deep learning-based open science tool for closed-loop brain stimulation
• Revisiting Tensor Basis Neural Networks for Reynolds stress modeling: application to plane channel and square duct flows
• Targeted Supervised Contrastive Learning for Long-Tailed Recognition
• Deep Extrapolation for Attribute-Enhanced Generation
• LeaFi: Data Series Indexes on Steroids with Learned Filters
• Score-balanced Loss for Multi-aspect Pronunciation Assessment
• Prototypical quadruplet for few-shot class incremental learning
• Technical Report for Valence-Arousal Estimation in ABAW2 Challenge