Modeling Language as a Sequence of Thoughts

Nasim Borazjanizadeh, James McClelland

Transformer language models can generate strikingly natural text by modeling language as a sequence of tokens, but by relying primarily on surface-level co-occurrence statistics th…