66 citations · 418 across the 62 of their papers we have counts for
85 papers
TimeCues Studio: A Workspace for Music Annotation and Algorithm Prototyping
Sapir Caduri, Yoav Goldberg
Multimedia applications require precise music annotation-labeled positions, segments, or loops-placed by hand or algorithmically. Machine-learning algorithms are scalable and effec…
MANCE: Manifold Aware Concept Erasure
Matan Avitan, Yoav Goldberg, Yanai Elazar
Concept erasure aims to remove a target concept from a representation while preserving the other information encoded in it. This is difficult because representations encode many co…
Diversity Over Quantity: A Lesson From Few Shot Relation Classification
Amir DN Cohen, Shauli Ravfogel, Shaltiel Shmidman +1
In few-shot relation classification (FSRC), models must generalize to novel relations with only a few labeled examples. While much of the recent progress in NLP has focused on scal…
Data-driven Coreference-based Ontology Building
Shir Ashury-Tahan, Amir David Nissan Cohen, Nadav Cohen +2
While coreference resolution is traditionally used as a component in individual document understanding, in this work we take a more global view and explore what can we learn about…
GRADE: Quantifying Sample Diversity in Text-to-Image Models
Royi Rassin, Aviv Slobodkin, Shauli Ravfogel +2
We introduce GRADE, an automatic method for quantifying sample diversity in text-to-image models. Our method leverages the world knowledge embedded in large language models and vis…
Data Contamination Report from the 2024 CONDA Shared Task
Oscar Sainz, Iker García-Ferrero, Alon Jacovi +25
The 1st Workshop on Data Contamination (CONDA 2024) focuses on all relevant aspects of data contamination in natural language processing, where data contamination is understood as…
A Practical Method for Generating String Counterfactuals
Matan Avitan, Ryan Cotterell, Yoav Goldberg +1
Interventions targeting the representation space of language models (LMs) have emerged as an effective means to influence model behavior. Such methods are employed, for example, to…
Guiding LLM to Fool Itself: Automatically Manipulating Machine Reading Comprehension Shortcut Triggers
Mosh Levy, Shauli Ravfogel, Yoav Goldberg
Recent applications of LLMs in Machine Reading Comprehension (MRC) systems have shown impressive results, but the use of shortcuts, mechanisms triggered by features spuriously corr…
Linguistic Binding in Diffusion Models: Enhancing Attribute Correspondence through Attention Map Alignment
Royi Rassin, Eran Hirsch, Daniel Glickman +3
Text-conditioned image generation models often generate incorrect associations between entities and their visual attributes. This reflects an impaired mapping between linguistic bi…
Clever Hans or Neural Theory of Mind? Stress Testing Social Reasoning in Large Language Models
Natalie Shapira, Mosh Levy, Seyed Hossein Alavi +5
The escalating debate on AI's capabilities warrants developing reliable metrics to assess machine "intelligence". Recently, many anecdotal examples were used to suggest that newer…
Neighboring Words Affect Human Interpretation of Saliency Explanations
Alon Jacovi, Hendrik Schuff, Heike Adel +2
Word-level saliency explanations ("heat maps over words") are often used to communicate feature-attribution in text-based models. Recent studies found that superficial factors such…
Conformal Nucleus Sampling
Shauli Ravfogel, Yoav Goldberg, Jacob Goldberger
Language models generate text based on successively sampling the next word. A decoding procedure based on nucleus (top-) sampling chooses from the smallest possible set of words…
Simple Linguistic Inferences of Large Language Models (LLMs): Blind Spots and Blinds
Victoria Basmov, Yoav Goldberg, Reut Tsarfaty
We evaluate LLMs' language understanding capacities on simple inference tasks that most humans find trivial. Specifically, we target (i) grammatically-specified entailments, (ii) p…
Description-Based Text Similarity
Shauli Ravfogel, Valentina Pyatkin, Amir DN Cohen +2
Identifying texts with a given semantics is central for many information seeking scenarios. Similarity search over vector embeddings appear to be central to this ability, yet the s…
Stop Uploading Test Data in Plain Text: Practical Strategies for Mitigating Data Contamination by Evaluation Benchmarks
Alon Jacovi, Avi Caciularu, Omer Goldman +1
Data contamination has become prevalent and challenging with the rise of models pretrained on large automatically-crawled corpora. For closed models, the training data becomes a tr…
Training Large Scale Polynomial CNNs for E2E Inference over Homomorphic Encryption
Moran Baruch, Nir Drucker, Gilad Ezov +5
Training large-scale CNNs that during inference can be run under Homomorphic Encryption (HE) is challenging due to the need to use only polynomial operations. This limits HE-based…
Two Kinds of Recall
Yoav Goldberg
It is an established assumption that pattern-based models are good at precision, while learning based models are better at recall. But is that really the case? I argue that there a…
At Your Fingertips: Extracting Piano Fingering Instructions from Videos
Amit Moryossef, Yanai Elazar, Yoav Goldberg
Piano fingering -- knowing which finger to use to play each note in a musical piece, is a hard and important skill to master when learning to play the piano. While some sheet music…
Lexical Generalization Improves with Larger Models and Longer Training
Elron Bandel, Yoav Goldberg, Yanai Elazar
While fine-tuned language models perform well on many tasks, they were also shown to rely on superficial surface features such as lexical overlap. Excessive utilization of such heu…
DALLE-2 is Seeing Double: Flaws in Word-to-Concept Mapping in Text2Image Models
Royi Rassin, Shauli Ravfogel, Yoav Goldberg
We study the way DALLE-2 maps symbols (words) in the prompt to their references (entities or properties of entities in the generated image). We show that in stark contrast to the w…
CIKQA: Learning Commonsense Inference with a Unified Knowledge-in-the-loop QA Paradigm
Hongming Zhang, Yintong Huo, Yanai Elazar +3
Recently, the community has achieved substantial progress on many commonsense reasoning benchmarks. However, it is still unclear what is learned from the training process: the know…
Log-linear Guardedness and its Implications
Shauli Ravfogel, Yoav Goldberg, Ryan Cotterell
Methods for erasing human-interpretable concepts from neural representations that assume linearity have been found to be tractable and useful. However, the impact of this removal o…
Understanding Transformer Memorization Recall Through Idioms
Adi Haviv, Ido Cohen, Jacob Gidron +3
To produce accurate predictions, language models (LMs) must balance between generalization and memorization. Yet, little is known about the mechanism by which transformer LMs emplo…
Measuring Causal Effects of Data Statistics on Language Model's `Factual' Predictions
Yanai Elazar, Nora Kassner, Shauli Ravfogel +6
Large amounts of training data are one of the major reasons for the high performance of state-of-the-art NLP models. But what exactly in the training data causes a model to make a…