6 papers
Mixing Mechanisms: How Language Models Retrieve Bound Entities In-Context
Yoav Gur-Arieh, Mor Geva, Atticus Geiger
A key component of in-context reasoning is the ability of language models (LMs) to bind entities for later retrieval. For example, an LM might represent "Ann loves pie" by binding…
Faithfulness Metrics Don't Measure Faithfulness: A Meta-Evaluation with Ground Truth
Yoav Gur-Arieh, Ana MarasoviÄ, Mor Geva
Chains of thought (CoTs) have become central in interpreting and auditing behaviors of large language models. Yet growing evidence suggests that these traces often fail to faithful…
Disentangling MLP Neuron Weights in Vocabulary Space
Asaf Avrahamy, Yoav Gur-Arieh, Mor Geva
Interpreting the information encoded in language model weights remains a fundamental challenge in mechanistic interpretability. In this work, we introduce ROTATE (Rotation-Optimize…
Precise In-Parameter Concept Erasure in Large Language Models
Yoav Gur-Arieh, Clara Suslik, Yihuai Hong +2
Large language models (LLMs) often acquire knowledge during pretraining that is undesirable in downstream deployments, e.g., sensitive information or copyrighted content. Existing…
LMEnt: A Suite for Analyzing Knowledge in Language Models from Pretraining Data to Representations
Daniela Gottesman, Alon Gilae-Dotan, Ido Cohen +4
Language models (LMs) increasingly drive real-world applications that require world knowledge. However, the internal processes through which models turn data into representations o…
Enhancing Automated Interpretability with Output-Centric Feature Descriptions
Yoav Gur-Arieh, Roy Mayan, Chen Agassy +2
Automated interpretability pipelines generate natural language descriptions for the concepts represented by features in large language models (LLMs), such as plants or the first wo…