Pretraining with Contrastive Sentence Objectives Improves Discourse Performance of Language Models

Dan Iter, Kelvin Guu, Larry Lansing +1

Recent models for unsupervised representation learning of text have employed a number of techniques to improve contextual word representations but have put little focus on discours…