Emotions are Subtle: Learning Sentiment Based Text Representations Using Contrastive Learning
arXiv:2112.01054
Abstract
Contrastive learning techniques have been widely used in the field of computer vision as a means of augmenting datasets. In this paper, we extend the use of these contrastive learning embeddings to sentiment analysis tasks and demonstrate that fine-tuning on these embeddings provides an improvement over fine-tuning on BERT-based embeddings to achieve higher benchmarks on the task of sentiment analysis when evaluated on the DynaSent dataset. We also explore how our fine-tuned models perform on cross-domain benchmark datasets. Additionally, we explore upsampling techniques to achieve a more balanced class distribution to make further improvements on our benchmark tasks.
arXiv comment9 pages, 5 figures
References in corpus (7)
- HuggingFace's Transformers: State-of-the-art Natural Language Processing
- Focal Loss for Dense Object Detection
- Whitening Sentence Representations for Better Semantics and Faster Retrieval
- ESimCSE: Enhanced Sample Building Method for Contrastive Learning of Unsupervised Sentence Embedding
- Contrastive Learning for Fair Representations
- Sentence Embeddings using Supervised Contrastive Learning
- Adversarial Training with Contrastive Learning in NLP