Invariant Atoms: Sparse Coordinates of Local Semantic Geometry in Language Model Representations

Muhammad Ahtesham, Xin Zhong

Large language models often preserve meaning despite substantial changes in wording, style, and syntax, while small semantic edits can systematically alter their hidden representat…