2 citations · 3 across the 9 of their papers we have counts for
9 papers
Can Agents Infer Environment from Interaction? Evidence from Agentic Automata Learning
Reef Menaged, Gili Lior, Shauli Ravfogel +2
We propose agentic automata learning to evaluate the extent to which tool-calling LLM agents can uncover hidden environments through interaction, a capability increasingly required…
Extending Item Response Theory for Efficient and Meaningful Multilingual Evaluation
Gili Lior, Tzviel Frostig, Gabriel Stanovsky +1
Multilingual benchmarks are central to evaluating large language models (LLMs) across languages, but they suffer from three issues: exhaustive evaluation scales linearly with the n…
PromptSuite: A Task-Agnostic Framework for Multi-Prompt Generation
Eliya Habba, Noam Dahan, Gili Lior +1
Evaluating LLMs with a single prompt has proven unreliable, with small changes leading to significant performance differences. However, generating the prompt variations needed for…
ReliableEval: A Recipe for Stochastic LLM Evaluation via Method of Moments
Gili Lior, Eliya Habba, Shahar Levy +2
LLMs are highly sensitive to prompt phrasing, yet standard benchmarks typically report performance using a single prompt, raising concerns about the reliability of such evaluations…
WildIFEval: Instruction Following in the Wild
Gili Lior, Asaf Yehudai, Ariel Gera +1
Recent LLMs have shown remarkable success in following user instructions, yet handling instructions with multiple constraints remains a significant challenge. In this work, we intr…
Comparing the Framing Effect in Humans and LLMs on Naturally Occurring Texts
Gili Lior, Liron Nacchace, Gabriel Stanovsky
Humans are influenced by how information is presented, a phenomenon known as the framing effect. Prior work suggests that LLMs may also be susceptible to framing, but it has relied…
SEAM: A Stochastic Benchmark for Multi-Document Tasks
Gili Lior, Avi Caciularu, Arie Cattan +3
Various tasks, such as summarization, multi-hop question answering, or coreference resolution, are naturally phrased over collections of real-world documents. Such tasks present a…
Leveraging Collection-Wide Similarities for Unsupervised Document Structure Extraction
Gili Lior, Yoav Goldberg, Gabriel Stanovsky
Document collections of various domains, e.g., legal, medical, or financial, often share some underlying collection-wide structure, which captures information that can aid both hum…
Comparing Humans and Models on a Similar Scale: Towards Cognitive Gender Bias Evaluation in Coreference Resolution
Gili Lior, Gabriel Stanovsky
Spurious correlations were found to be an important factor explaining model performance in various NLP tasks (e.g., gender or racial artifacts), often considered to be ''shortcuts'…