Contrastive Mixup: Self- and Semi-Supervised learning for Tabular Domain

Abstract

Recent literature in self-supervised has demonstrated significant progress in closing the gap between supervised and unsupervised methods in the image and text domains. These methods rely on domain-specific augmentations that are not directly amenable to the tabular domain. Instead, we introduce Contrastive Mixup, a semi-supervised learning framework for tabular data and demonstrate its effectiveness in limited annotated data settings. Our proposed method leverages Mixup-based augmentation under the manifold assumption by mapping samples to a low dimensional latent space and encourage interpolated samples to have high a similarity within the same labeled class. Unlabeled samples are additionally employed via a transductive label propagation method to further enrich the set of similar and dissimilar pairs that can be used in the contrastive loss term. We demonstrate the effectiveness of the proposed framework on public tabular datasets and real-world clinical datasets.

References in corpus (8)

• A Simple Framework for Contrastive Learning of Visual Representations
• mixup: Beyond Empirical Risk Minimization
• Mean teachers are better role models: Weight-averaged consistency targets improve semi-supervised deep learning results
• CatBoost: unbiased boosting with categorical features
• Manifold Mixup: Better Representations by Interpolating Hidden States
• An efficient framework for learning sentence representations
• Supervised Contrastive Learning
• TaBERT: Pretraining for Joint Understanding of Textual and Tabular Data