21 citations · 70 across the 16 of their papers we have counts for
14 papers · 1 filter
Self-Execution Simulation Improves Coding Models
Gallil Maimon, Ori Yoran, Felix Kreuk +4
A promising research direction in enabling LLMs to generate consistently correct code involves addressing their inability to properly estimate program execution, particularly for c…
LMEnt: A Suite for Analyzing Knowledge in Language Models from Pretraining Data to Representations
Daniela Gottesman, Alon Gilae-Dotan, Ido Cohen +4
Language models (LMs) increasingly drive real-world applications that require world knowledge. However, the internal processes through which models turn data into representations o…
The KoLMogorov Test: Compression by Code Generation
Ori Yoran, Kunhao Zheng, Fabian Gloeckle +3
Compression is at the heart of intelligence. A theoretically optimal way to compress any sequence of data is to find the shortest program that outputs that sequence and then halts.…
Preventing Rogue Agents Improves Multi-Agent Collaboration
Ohav Barbi, Ori Yoran, Mor Geva
Multi-agent systems, where specialized agents collaborate to solve a shared task hold great potential, from increased modularity to simulating complex environments. However, they a…
AssistantBench: Can Web Agents Solve Realistic and Time-Consuming Tasks?
Ori Yoran, Samuel Joseph Amouyal, Chaitanya Malaviya +3
Language agents, built on top of language models (LMs), are systems that can interact with complex environments, such as the open web. In this work, we examine whether such agents…
From Loops to Oops: Fallback Behaviors of Language Models Under Uncertainty
Maor Ivgi, Ori Yoran, Jonathan Berant +1
Large language models (LLMs) often exhibit undesirable behaviors, such as hallucinations and sequence repetitions. We propose to view these behaviors as fallbacks that models exhib…
Making Retrieval-Augmented Language Models Robust to Irrelevant Context
Ori Yoran, Tomer Wolfson, Ori Ram +1
Retrieval-augmented language models (RALMs) hold promise to produce language understanding systems that are are factual, efficient, and up-to-date. An important desideratum of RALM…
Evaluating the Ripple Effects of Knowledge Editing in Language Models
Roi Cohen, Eden Biran, Ori Yoran +2
Modern language models capture a large body of factual knowledge. However, some facts can be incorrectly induced or become obsolete over time, resulting in factually incorrect gene…
Answering Questions by Meta-Reasoning over Multiple Chains of Thought
Ori Yoran, Tomer Wolfson, Ben Bogin +3
Modern systems for multi-hop question answering (QA) typically break questions into a sequence of reasoning steps, termed chain-of-thought (CoT), before arriving at a final answer.…
QAMPARI: An Open-domain Question Answering Benchmark for Questions with Many Answers from Multiple Paragraphs
Samuel Joseph Amouyal, Tomer Wolfson, Ohad Rubin +3
Existing benchmarks for open-domain question answering (ODQA) typically focus on questions whose answers can be extracted from a single paragraph. By contrast, many natural questio…
CommonsenseQA 2.0: Exposing the Limits of AI through Gamification
Alon Talmor, Ori Yoran, Ronan Le Bras +4
Constructing benchmarks that test the abilities of modern natural language understanding models is difficult - pre-trained language models exploit artifacts in benchmarks to achiev…
SCROLLS: Standardized CompaRison Over Long Language Sequences
Uri Shaham, Elad Segal, Maor Ivgi +8
NLP benchmarks have largely focused on short texts, such as sentences and paragraphs, even though long texts comprise a considerable amount of natural language in the wild. We intr…
Turning Tables: Generating Examples from Semi-structured Tables for Endowing Language Models with Reasoning Skills
Ori Yoran, Alon Talmor, Jonathan Berant
Models pre-trained with a language modeling objective possess ample world knowledge and language skills, but are known to struggle in tasks that require reasoning. In this work, we…
MultiModalQA: Complex Question Answering over Text, Tables and Images
Alon Talmor, Ori Yoran, Amnon Catav +6
When answering complex questions, people can seamlessly combine information from visual, textual and tabular sources. While interest in models that reason over multiple pieces of e…