565 citations · 734 across the 71 of their papers we have counts for
13 papers · 2 filters
Pretraining Latent Information Feedback Transformers with Teacher Supervision
Dor Tirosh, Ido Amos, Mor Geva
Transformer language models (LMs) are feed-forward: deep-layer representations are never fed back to shallower layers, and the only pathway for information to flow downward across…
Capacity, Responsiveness and Alignment: What Makes a Latent Structure Actionable
Or Shafran, Mor Geva
Localizing latent structures in the activation space of language models (LMs) is central to understanding and controlling their behavior. Yet, localized structures can differ subst…
LMs as Task-Specific Knowledge Bases: An Interpretability Analysis
Amit Elhelo, Amir Globerson, Mor Geva
Language models (LMs) capture large amounts of factual knowledge applicable to a wide range of tasks, motivating the view of their parameters as a knowledge base. An important prop…
Don't Forget Your Embeddings: Robust Knowledge Erasure via Precise Editing of Embeddings
Clara Haya Suslik, Or Shafran, Mor Geva
As language models are increasingly deployed in real-world applications, the ability to erase specific knowledge from them becomes critical for safety and compliance. Prominent met…
Faithfulness Metrics Don't Measure Faithfulness: A Meta-Evaluation with Ground Truth
Yoav Gur-Arieh, Ana Marasović, Mor Geva
Chains of thought (CoTs) have become central in interpreting and auditing behaviors of large language models. Yet growing evidence suggests that these traces often fail to faithful…
Friends and Grandmothers in Silico: Localizing Entity Cells in Language Models
Itay Yona, Dan Barzilay, Michael Karasik +1
How do language models retrieve entity-specific facts from their parameters? We investigate this question by searching for sparse, entity-selective MLP neurons - which we call enti…