4 citations · 8 across the 10 of their papers we have counts for
12 papers
Stop Guessing When to Stop Testing: Efficient Model Evaluation with Just Enough Data
Ofir Arviv, Kristjan Greenewald, Yotam Perlitz +3
The inherent rigidity of fixed-size benchmarks makes them an inefficient tool for model evaluation. Diverse evaluation objectives, including model ranking, model selection and test…
DOVE: A Large-Scale Multi-Dimensional Predictions Dataset Towards Meaningful LLM Evaluation
Eliya Habba, Ofir Arviv, Itay Itzhak +5
Recent work found that LLMs are sensitive to a wide range of arbitrary prompt dimensions, including the type of delimiters, answer enumerators, instruction wording, and more. This…
Stay Tuned: An Empirical Study of the Impact of Hyperparameters on LLM Tuning in Real-World Applications
Alon Halfon, Shai Gretz, Ofir Arviv +6
Fine-tuning Large Language Models (LLMs) is an effective method to enhance their performance on downstream tasks. However, choosing the appropriate setting of tuning hyperparameter…
Do These LLM Benchmarks Agree? Fixing Benchmark Evaluation with BenchBench
Yotam Perlitz, Ariel Gera, Ofir Arviv +5
Recent advancements in Language Models (LMs) have catalyzed the creation of multiple benchmarks, designed to assess these models' general capabilities. A crucial task, however, is…
Genie: Achieving Human Parity in Content-Grounded Datasets Generation
Asaf Yehudai, Boaz Carmeli, Yosi Mass +5
The lack of high-quality data for content-grounded generation tasks has been identified as a major obstacle to advancing these tasks. To address this gap, we propose Genie, a novel…
Unitxt: Flexible, Shareable and Reusable Data Preparation and Evaluation for Generative AI
Elron Bandel, Yotam Perlitz, Elad Venezian +9
In the dynamic landscape of generative NLP, traditional text processing pipelines limit research flexibility and reproducibility, as they are tailored to specific dataset, task, an…
Improving Cross-Lingual Transfer through Subtree-Aware Word Reordering
Ofir Arviv, Dmitry Nikolaev, Taelin Karidi +1
Despite the impressive growth of the abilities of multilingual language models, such as XLM-R and mT5, it has been shown that they still face difficulties when tackling typological…
Efficient Benchmarking of Language Models
Yotam Perlitz, Elron Bandel, Ariel Gera +6
The increasing versatility of language models (LMs) has given rise to a new class of benchmarks that comprehensively assess a broad range of capabilities. Such benchmarks are assoc…
The Benefits of Bad Advice: Autocontrastive Decoding across Model Layers
Ariel Gera, Roni Friedman, Ofir Arviv +4
Applying language models to natural language processing tasks typically relies on the representations in the final model layer, as intermediate hidden layer representations are pre…
On the Relation between Syntactic Divergence and Zero-Shot Performance
Ofir Arviv, Dmitry Nikolaev, Taelin Karidi +1
We explore the link between the extent to which syntactic relations are preserved in translation and the ease of correctly constructing a parse tree in a zero-shot setting. While p…
HUJI-KU at MRP~2020: Two Transition-based Neural Parsers
Ofir Arviv, Ruixiang Cui, Daniel Hershcovich
This paper describes the HUJI-KU system submission to the shared task on Cross-Framework Meaning Representation Parsing (MRP) at the 2020 Conference for Computational Language Lear…
Fine-Grained Analysis of Cross-Linguistic Syntactic Divergences
Dmitry Nikolaev, Ofir Arviv, Taelin Karidi +4
The patterns in which the syntax of different languages converges and diverges are often used to inform work on cross-lingual transfer. Nevertheless, little empirical work has been…