How Pre-trained Language Models Capture Factual Knowledge? A Causal-Inspired Analysis

Shaobo Li, Xiaoguang Li, Lifeng Shang +6

Recently, there has been a trend to investigate the factual knowledge captured by Pre-trained Language Models (PLMs). Many works show the PLMs' ability to fill in the missing factu…