Mechanistic Understanding and Mitigation of Language Model Non-Factual Hallucinations

Lei Yu, Meng Cao, Jackie Chi Kit Cheung +1

State-of-the-art language models (LMs) sometimes generate non-factual hallucinations that misalign with world knowledge. To explore the mechanistic causes of these hallucinations,…