8 papers
Factual Retrieval in LLMs Is a Redundant, Distributed and Non-Contiguous Process
Hail Hochman, Natalie Shapira, Yoav Goldberg
Large language models (LLMs) store and recall factual knowledge, yet the precise mechanism of how entity representations are transformed to enable specific attribute retrieval rema…
Forecasting Future Behavior as a Learning Task
Mosh Levy, Yoav Goldberg, Asa Cooper Stickland
Trust in an AI system is often anchored by explanations of how it works, which one then uses to forecast its behavior on new inputs. For large reasoning models (LRMs), this convent…
Compared to What? Baselines and Metrics for Counterfactual Prompting
Zihao Yang, Mosh Levy, Yoav Goldberg +1
Counterfactual prompting (i.e., perturbing a single factor and measuring output change) is widely used to evaluate things like LLM bias and CoT faithfulness. But in this work we ar…
Understanding Usage and Engagement in AI-Powered Scientific Research Tools: The Asta Interaction Dataset
Dany Haddad, Dan Bareket, Joseph Chee Chang +19
AI-powered scientific research tools are rapidly being integrated into research workflows, yet the field lacks a clear lens into how researchers use these systems in real-world set…
State over Tokens: Characterizing the Role of Reasoning Tokens
Mosh Levy, Zohar Elyoseph, Shauli Ravfogel +1
Large Language Models (LLMs) can generate reasoning tokens before their final answer to boost performance on complex tasks. While these sequences seem like human thought processes,…
NER Retriever: Zero-Shot Named Entity Retrieval with Type-Aware Embeddings
Or Shachar, Uri Katz, Yoav Goldberg +1
We present NER Retriever, a zero-shot retrieval framework for ad-hoc Named Entity Retrieval, a variant of Named Entity Recognition (NER), where the types of interest are not provid…