2 citations · 7 across the 14 of their papers we have counts for
8 papers · 1 filter
Last Translation Benchmark
Vilém Zouhar, Niyati Bafna, Mukund Choudhary +263
For scientific progress, we need benchmarks that test the limits of state-of-the-art models, and evaluation methods that inform us about failure cases. As models get stronger, stan…
From Feelings to Metrics: Understanding and Formalizing How Users Vibe-Test LLMs
Itay Itzhak, Eliya Habba, Gabriel Stanovsky +1
Evaluating LLMs is challenging, as benchmark scores often fail to capture models' real-world usefulness. Instead, users often rely on ``vibe-testing'': informal experience-based ev…
ScheMatiQ: From Research Question to Structured Data through Interactive Schema Discovery
Shahar Levy, Eliya Habba, Reshef Mintz +3
Many disciplines pose natural-language research questions over large document collections whose answers typically require structured evidence, traditionally obtained by manually de…
JSON Whisperer: Efficient JSON Editing with LLMs
Sarel Duanis, Asnat Greenstein-Messica, Eliya Habba
Large language models (LLMs) can modify JSON documents through natural language commands, but current approaches regenerate entire structures for each edit, resulting in computatio…
PromptSuite: A Task-Agnostic Framework for Multi-Prompt Generation
Eliya Habba, Noam Dahan, Gili Lior +1
Evaluating LLMs with a single prompt has proven unreliable, with small changes leading to significant performance differences. However, generating the prompt variations needed for…
ReliableEval: A Recipe for Stochastic LLM Evaluation via Method of Moments
Gili Lior, Eliya Habba, Shahar Levy +2
LLMs are highly sensitive to prompt phrasing, yet standard benchmarks typically report performance using a single prompt, raising concerns about the reliability of such evaluations…
DOVE: A Large-Scale Multi-Dimensional Predictions Dataset Towards Meaningful LLM Evaluation
Eliya Habba, Ofir Arviv, Itay Itzhak +5
Recent work found that LLMs are sensitive to a wide range of arbitrary prompt dimensions, including the type of delimiters, answer enumerators, instruction wording, and more. This…
The Perfect Victim: Computational Analysis of Judicial Attitudes towards Victims of Sexual Violence
Eliya Habba, Renana Keydar, Dan Bareket +1
We develop computational models to analyze court statements in order to assess judicial attitudes toward victims of sexual violence in the Israeli court system. The study examines…