Shuffle & Divide: Contrastive Learning for Long Text

Joonseok Lee, Seongho Joe, Kyoungwon Park +4

We propose a self-supervised learning method for long text documents based on contrastive learning. A key to our method is Shuffle and Divide (SaD), a simple text augmentation algo…