Can You Learn Semantics Through Next-Word Prediction? The Case of Entailment

William Merrill, Zhaofeng Wu, Norihito Naka +2

Do LMs infer the semantics of text from co-occurrence patterns in their training data? Merrill et al. (2022) argue that, in theory, sentence co-occurrence probabilities predicted b…