Factual Retrieval in LLMs Is a Redundant, Distributed and Non-Contiguous Process

Hail Hochman, Natalie Shapira, Yoav Goldberg

Large language models (LLMs) store and recall factual knowledge, yet the precise mechanism of how entity representations are transformed to enable specific attribute retrieval rema…