What Has Been Enhanced in my Knowledge-Enhanced Language Model?

Yifan Hou, Guoji Fu, Mrinmaya Sachan

Pretrained language models (LMs) do not capture factual knowledge very well. This has led to the development of a number of knowledge integration (KI) methods which aim to incorpor…