Exploring the Knowledge Mismatch Hypothesis: Hallucination Propensity in Small Models Fine-tuned on Data from Larger Models

Phil Wee, Riyadh Baghdadi

Recently, there has been an explosion of large language models created through fine-tuning with data from larger models. These small models able to produce outputs that appear qual…