37 citations · 46 across the 15 of their papers we have counts for
17 papers
Skill Issue: Are Skills Language-Invariant in LLMs?
Bobby Cheng, Adam Gaber, Zhengyuan Liu +4
Large language models access knowledge inconsistently across languages, but to what extent do they differ in their skill sets when interacting with different languages? This work q…
MameLoshnLM: Yiddish Language Model and Evaluation Benchmark
Uri Katz, Omer Goldman, Tomasz Limisiewicz +2
We present MameLoshnLM, the first open-source 8B-parameter language model built specifically for Yiddish. Despite Yiddish's rich textual tradition, its limited digital presence and…
Location Not Found: Exposing Implicit Local and Global Biases in Multilingual LLMs
Guy Mor-Lan, Omer Goldman, Matan Eyal +6
Multilingual large language models (LLMs) have minimized the fluency gap between languages. This advancement, however, exposes models to the risk of biased behavior, as knowledge a…
ECLeKTic: a Novel Challenge Set for Evaluation of Cross-Lingual Knowledge Transfer
Omer Goldman, Uri Shaham, Dan Malkin +11
To achieve equitable performance across languages, large language models (LLMs) must be able to abstract knowledge beyond the language in which it was learnt. However, the current…
Is It Really Long Context if All You Need Is Retrieval? Towards Genuinely Difficult Long Context NLP
Omer Goldman, Alon Jacovi, Aviv Slobodkin +3
Improvements in language models' capabilities have pushed their applications towards longer contexts, making long-context evaluation and development an active research area. Howeve…
Unpacking Tokenization: Evaluating Text Compression and its Correlation with Model Performance
Omer Goldman, Avi Caciularu, Matan Eyal +3
Despite it being the cornerstone of BPE, the most common tokenization algorithm, the importance of compression in the tokenization process is still unclear. In this paper, we argue…
The Curious Case of Hallucinatory (Un)answerability: Finding Truths in the Hidden States of Over-Confident Large Language Models
Aviv Slobodkin, Omer Goldman, Avi Caciularu +2
Large language models (LLMs) have been shown to possess impressive capabilities, while also raising crucial concerns about the faithfulness of their responses. A primary issue aris…
Explicit Morphological Knowledge Improves Pre-training of Language Models for Hebrew
Eylon Gueta, Omer Goldman, Reut Tsarfaty
Pre-trained language models (PLMs) have shown remarkable successes in acquiring a wide range of linguistic knowledge, relying solely on self-supervised training on text streams. Ne…
Is Probing All You Need? Indicator Tasks as an Alternative to Probing Embedding Spaces
Tal Levy, Omer Goldman, Reut Tsarfaty
The ability to identify and control different kinds of linguistic information encoded in vector representations of words has many use cases, especially for explainability and bias…
Morphological Inflection with Phonological Features
David Guriel, Omer Goldman, Reut Tsarfaty
Recent years have brought great advances into solving morphological tasks, mostly due to powerful neural models applied to various tasks as (re)inflection and analysis. Yet, such m…
Stop Uploading Test Data in Plain Text: Practical Strategies for Mitigating Data Contamination by Evaluation Benchmarks
Alon Jacovi, Avi Caciularu, Omer Goldman +1
Data contamination has become prevalent and challenging with the rise of models pretrained on large automatically-crawled corpora. For closed models, the training data becomes a tr…
UniMorph 4.0: Universal Morphology
Khuyagbaatar Batsuren, Omer Goldman, Salam Khalifa +93
The Universal Morphology (UniMorph) project is a collaborative effort providing broad-coverage instantiated normalized morphological inflection tables for hundreds of diverse world…
Morphological Reinflection with Multiple Arguments: An Extended Annotation schema and a Georgian Case Study
David Guriel, Omer Goldman, Reut Tsarfaty
In recent years, a flurry of morphological datasets had emerged, most notably UniMorph, a multi-lingual repository of inflection tables. However, the flat structure of the current…
Morphology Without Borders: Clause-Level Morphology
Omer Goldman, Reut Tsarfaty
Morphological tasks use large multi-lingual datasets that organize words into inflection tables, which then serve as training and evaluation data for various tasks. However, a clos…
(Un)solving Morphological Inflection: Lemma Overlap Artificially Inflates Models' Performance
Omer Goldman, David Guriel, Reut Tsarfaty
In the domain of Morphology, Inflection is a fundamental and important task that gained a lot of traction in recent years, mostly via SIGMORPHON's shared-tasks. With average accura…
Minimal Supervision for Morphological Inflection
Omer Goldman, Reut Tsarfaty
Neural models for the various flavours of morphological inflection tasks have proven to be extremely accurate given ample labeled data -- data that may be slow and costly to obtain…
Weakly-supervised Semantic Parsing with Abstract Examples
Omer Goldman, Veronica Latcinnik, Udi Naveh +2
Training semantic parsers from weak supervision (denotations) rather than strong supervision (programs) complicates training in two ways. First, a large search space of potential p…