4 papers
Knowing When to Quit: A Principled Framework for Dynamic Abstention in LLM Reasoning
Hen Davidov, Nachshon Cohen, Oren Kalinsky +4
LLMs utilizing chain-of-thought reasoning often waste substantial compute by producing long, incorrect responses. Abstention can mitigate this by withholding outputs unlikely to be…
HotelQuEST: Balancing Quality and Efficiency in Agentic Search
Guy Hadad, Shadi Iskander, Oren Kalinsky +3
Agentic search has emerged as a promising paradigm for adaptive retrieval systems powered by large language models (LLMs). However, existing benchmarks primarily focus on quality,…
ChaI-TeA: A Benchmark for Evaluating Autocompletion of Interactions with LLM-based Chatbots
Shani Goren, Oren Kalinsky, Tomer Stav +6
The rise of LLMs has deflected a growing portion of human-computer interactions towards LLM-based chatbots. The remarkable abilities of these models allow users to interact using l…
Evaluating D-MERIT of Partial-annotation on Information Retrieval
Royi Rassin, Yaron Fairstein, Oren Kalinsky +4
Retrieval models are often evaluated on partially-annotated datasets. Each query is mapped to a few relevant texts and the remaining corpus is assumed to be irrelevant. As a result…