Dynamic Weight Grafting: Localizing Finetuned Factual Knowledge in Transformers

Todd Nief, David Reber, Sean Richardson +1

When an LLM learns a new fact during finetuning (e.g., new movie releases, newly elected pope, etc.), where does this information go? Are entities enriched with relation informatio…