Cross-Domain Sentiment Classification with In-Domain Contrastive Learning
arXiv:2012.02943
Abstract
Contrastive learning (CL) has been successful as a powerful representation learning method. In this paper, we propose a contrastive learning framework for cross-domain sentiment classification. We aim to induce domain invariant optimal classifiers rather than distribution matching. To this end, we introduce in-domain contrastive learning and entropy minimization. Also, we find through ablation studies that these two techniques behaviour differently in case of large label distribution shift and conclude that the best practice is to choose one of them adaptively according to label distribution shift. The new state-of-the-art results our model achieves on standard benchmarks show the efficacy of the proposed method.
arXiv comment10pages, 2 figures, accepted to NeurIPS 2020 Workshop on Self-supervised Learning. arXiv admin note: text overlap with arXiv:2010.16088
References in corpus (15)
- A Simple Framework for Contrastive Learning of Visual Representations
- HuggingFace's Transformers: State-of-the-art Natural Language Processing
- Improved Baselines with Momentum Contrastive Learning
- Momentum Contrast for Unsupervised Visual Representation Learning
- Equivalence of distance-based and RKHS-based statistics in hypothesis testing
- Big Self-Supervised Models are Strong Semi-Supervised Learners
- Tent: Fully Test-time Adaptation by Entropy Minimization
- Unsupervised Domain Adaptation through Self-Supervision
- On Learning Invariant Representation for Domain Adaptation
- Many Paths to Equilibrium: GANs Do Not Need to Decrease a Divergence At Every Step
- DeCLUTR: Deep Contrastive Learning for Unsupervised Textual Representations
- Domain Adaptation with Conditional Distribution Matching and Generalized Label Shift
- Rethinking Distributional Matching Based Domain Adaptation
- Invariant Risk Minimization Games
- Adaptive Semi-supervised Learning for Cross-domain Sentiment Classification