activity
20192021
most citedA Primer on Contrastive Pretraining in Language Processing: Methods, Lessons Learned and Perspectives

2 citations · 2 across the 1 of their papers we have counts for

collaborators
Showing cs.CLShow all

2 papers · 1 filter

cs.CL2021★ 2 cited

A Primer on Contrastive Pretraining in Language Processing: Methods, Lessons Learned and Perspectives

Nils Rethmeier, Isabelle Augenstein

Modern natural language processing (NLP) methods employ self-supervised pretraining objectives such as masked language modeling to boost the performance of various application task…

cs.CL2020

Data-Efficient Pretraining via Contrastive Self-Supervision

Nils Rethmeier, Isabelle Augenstein

For natural language processing `text-to-text' tasks, the prevailing approaches heavily rely on pretraining large self-supervised models on increasingly larger `task-external' data…