SenseBERT: Driving Some Sense into BERT

Yoav Levine, Barak Lenz, Or Dagan +6

The ability to learn from large unlabeled corpora has allowed neural language models to advance the frontier in natural language understanding. However, existing self-supervision t…