12 citations · 28 across the 9 of their papers we have counts for
11 papers
PolySQL: Scaling Text-to-SQL Evaluation Across SQL Dialects via Automated Backend Isomorphism
Yotam Perlitz, Elad Venezian, Corentin Royer +2
SQL dialects vary in syntax, types, and functions across database engines. Text-to-SQL benchmarks, however, predominantly support only SQLite. This creates a critical evaluation ga…
General Agent Evaluation
Elron Bandel, Asaf Yehudai, Lilach Eden +12
General-purpose agents perform tasks in unfamiliar environments without domain-specific manual customization. Yet no study has systematically measured how agent architecture shapes…
Unitxt: Flexible, Shareable and Reusable Data Preparation and Evaluation for Generative AI
Elron Bandel, Yotam Perlitz, Elad Venezian +9
In the dynamic landscape of generative NLP, traditional text processing pipelines limit research flexibility and reproducibility, as they are tailored to specific dataset, task, an…
Knowledge is a Region in Weight Space for Fine-tuned Language Models
Almog Gueta, Elad Venezian, Colin Raffel +3
Research on neural networks has focused on understanding a single model trained on a single dataset. However, relatively little is known about the relationships between different m…
ColD Fusion: Collaborative Descent for Distributed Multitask Finetuning
Shachar Don-Yehiya, Elad Venezian, Colin Raffel +3
We propose a new paradigm to continually evolve pretrained models, denoted ColD Fusion. It provides the benefits of multitask learning but leverages distributed computation with li…
Where to start? Analyzing the potential value of intermediate models
Leshem Choshen, Elad Venezian, Shachar Don-Yehia +2
Previous studies observed that finetuned models may be better base models than the vanilla pretrained model. Such a model, finetuned on some source dataset, may provide a better st…
Fusing finetuned models for better pretraining
Leshem Choshen, Elad Venezian, Noam Slonim +1
Pretrained models are the standard starting point for training. This approach consistently outperforms the use of a random initialization. However, pretraining is a costly endeavou…
Project Debater APIs: Decomposing the AI Grand Challenge
Roy Bar-Haim, Yoav Kantor, Elad Venezian +2
Project Debater was revealed in 2019 as the first AI system that can debate human experts on complex topics. Engaging in a live debate requires a diverse set of skills, and Project…
Automatic Argument Quality Assessment -- New Datasets and Methods
Assaf Toledo, Shai Gretz, Edo Cohen-Karlik +6
We explore the task of automatic assessment of argument quality. To that end, we actively collected 6.3k arguments, more than a factor of five compared to previously examined data.…
What did you Mention? A Large Scale Mention Detection Benchmark for Spoken and Written Text
Yosi Mass, Lili Kotlerman, Shachar Mirkin +3
We describe a large, high-quality benchmark for the evaluation of Mention Detection tools. The benchmark contains annotations of both named entities as well as other types of entit…
A Recorded Debating Dataset
Shachar Mirkin, Michal Jacovi, Tamar Lavee +6
This paper describes an English audio and textual dataset of debating speeches, a unique resource for the growing research field of computational argumentation and debating technol…