6 papers · 1 filter
Performance Gap in Entity Knowledge Extraction Across Modalities in Vision Language Models
Ido Cohen, Daniela Gottesman, Mor Geva +1
Vision-language models (VLMs) excel at extracting and reasoning about information from images. Yet, their capacity to leverage internal knowledge about specific entities remains un…
How Well Can Reasoning Models Identify and Recover from Unhelpful Thoughts?
Sohee Yang, Sang-Woo Lee, Nora Kassner +3
Recent reasoning models show the ability to reflect, backtrack, and self-validate their reasoning, which is crucial in spotting mistakes and arriving at accurate solutions. A natur…
LMEnt: A Suite for Analyzing Knowledge in Language Models from Pretraining Data to Representations
Daniela Gottesman, Alon Gilae-Dotan, Ido Cohen +4
Language models (LMs) increasingly drive real-world applications that require world knowledge. However, the internal processes through which models turn data into representations o…
Estimating Knowledge in Large Language Models Without Generating a Single Token
Daniela Gottesman, Mor Geva
To evaluate knowledge in large language models (LLMs), current methods query the model and then evaluate its generated responses. In this work, we ask whether evaluation can be don…
Eliciting Textual Descriptions from Representations of Continuous Prompts
Dana Ramati, Daniela Gottesman, Mor Geva
Continuous prompts, or "soft prompts", are a widely-adopted parameter-efficient tuning strategy for large language models, but are often less favorable due to their opaque nature.…
Hopping Too Late: Exploring the Limitations of Large Language Models on Multi-Hop Queries
Eden Biran, Daniela Gottesman, Sohee Yang +2
Large language models (LLMs) can solve complex multi-step problems, but little is known about how these computations are implemented internally. Motivated by this, we study how LLM…