A Survey of Knowledge Enhanced Pre-trained Models

Jian Yang, Xinyu Hu, Gang Xiao +1

Pre-trained language models learn informative word representations on a large-scale text corpus through self-supervised learning, which has achieved promising performance in fields…