The Curious Case of Factuality Finetuning: Models' Internal Beliefs Can Improve Factuality

Benjamin Newman, Abhilasha Ravichander, Jaehun Jung +5

Language models are prone to hallucination - generating text that is factually incorrect. Finetuning models on high-quality factual information can potentially reduce hallucination…