8 papers
The Overlooked Role of Graded Relevance Thresholds in Multilingual Dense Retrieval
Tomer Wullach, Ori Shapira, Amir DN Cohen
Dense retrieval models are typically fine-tuned with contrastive learning objectives that require binary relevance judgments, even though relevance is inherently graded. We analyze…
Consensus or Conflict? Fine-Grained Evaluation of Conflicting Answers in Question-Answering
Eviatar Nachshoni, Arie Cattan, Shmuel Amar +2
Large Language Models (LLMs) have demonstrated strong performance in question answering (QA) tasks. However, Multi-Answer Question Answering (MAQA), where a question may have sever…
A Unifying Scheme for Extractive Content Selection Tasks
Shmuel Amar, Ori Shapira, Aviv Slobodkin +1
A broad range of NLP tasks involve selecting relevant text spans from given source texts. Despite this shared objective, such \textit{content selection} tasks have traditionally be…
Information Types in Product Reviews
Ori Shapira, Yuval Pinter
Information in text is communicated in a way that supports a goal for its reader. Product reviews, for example, contain opinions, tips, product descriptions, and many other types o…
Measuring the Effect of Transcription Noise on Downstream Language Understanding Tasks
Ori Shapira, Shlomo E. Chazan, Amir DN Cohen
With the increasing prevalence of recorded human speech, spoken language understanding (SLU) is essential for its efficient processing. In order to process the speech, it is common…
Quality Matters: Evaluating Synthetic Data for Tool-Using LLMs
Shadi Iskander, Nachshon Cohen, Zohar Karnin +2
Training large language models (LLMs) for external tool usage is a rapidly expanding field, with recent research focusing on generating synthetic data to address the shortage of av…