Latent Knowledge as a Predictor of Fact Acquisition in Fine-Tuned Large Language Models

Daniel B. Hier, Tayo Obafemi-Ajayi

Large language models store biomedical facts with uneven strength after pretraining: some facts are present in the weights but are not reliably accessible under deterministic decod…