66 citations · 418 across the 62 of their papers we have counts for
18 papers · 1 filter
Simple, Interpretable and Stable Method for Detecting Words with Usage Change across Corpora
Hila Gonen, Ganesh Jawahar, Djamé Seddah +1
The problem of comparing two bodies of text and searching for words that differ in their usage between them arises often in digital humanities and computational social science. Thi…
Large Scale Substitution-based Word Sense Induction
Matan Eyal, Shoval Sadde, Hillel Taub-Tabib +1
We present a word-sense induction method based on pre-trained masked language models (MLMs), which can cheaply scale to large vocabularies and large corpora. The result is a corpus…
Asking It All: Generating Contextualized Questions for any Semantic Role
Valentina Pyatkin, Paul Roit, Julian Michael +3
Asking questions about a situation is an inherent step towards understanding it. To this end, we introduce the task of role question generation, which, given a predicate mention an…
Text-based NP Enrichment
Yanai Elazar, Victoria Basmov, Yoav Goldberg +1
Understanding the relations between entities denoted by NPs in a text is a critical part of human-like natural language understanding. However, only a fraction of such relations is…
Thinking Like Transformers
Gail Weiss, Yoav Goldberg, Eran Yahav
What is the computational model behind a Transformer? Where recurrent neural networks have direct parallels in finite state machines, allowing clear discussion and thought around a…
Neural Extractive Search
Shauli Ravfogel, Hillel Taub-Tabib, Yoav Goldberg
Domain experts often need to extract structured information from large corpora. We advocate for a search paradigm called ``extractive search'', in which a search query is enriched…
BitFit: Simple Parameter-efficient Fine-tuning for Transformer-based Masked Language-models
Elad Ben-Zaken, Shauli Ravfogel, Yoav Goldberg
We introduce BitFit, a sparse-finetuning method where only the bias-terms of the model (or a subset of them) are being modified. We show that with small-to-medium training data, ap…
Data Augmentation for Sign Language Gloss Translation
Amit Moryossef, Kayo Yin, Graham Neubig +1
Sign language translation (SLT) is often decomposed into video-to-gloss recognition and gloss-to-text translation, where a gloss is a sequence of transcribed spoken-language words…
Counterfactual Interventions Reveal the Causal Effect of Relative Clause Representations on Agreement Prediction
Shauli Ravfogel, Grusha Prasad, Tal Linzen +1
When language models process syntactically complex sentences, do they use their representations of syntax in a manner that is consistent with the grammar of the language? We propos…
Including Signed Languages in Natural Language Processing
Kayo Yin, Amit Moryossef, Julie Hochgesang +2
Signed languages are the primary means of communication for many deaf and hard of hearing individuals. Since signed languages exhibit all the fundamental linguistic properties of n…
Does BERT Pretrained on Clinical Notes Reveal Sensitive Data?
Eric Lehman, Sarthak Jain, Karl Pichotta +2
Large Transformers pretrained over clinical notes from Electronic Health Records (EHR) have afforded substantial gains in performance on predictive clinical tasks. The cost of trai…
Revisiting Few-shot Relation Classification: Evaluation Data and Classification Schemes
Ofer Sabo, Yanai Elazar, Yoav Goldberg +1
We explore Few-Shot Learning (FSL) for Relation Classification (RC). Focusing on the realistic scenario of FSL, in which a test instance might not belong to any of the target categ…
Provable Limitations of Acquiring Meaning from Ungrounded Form: What Will Future Language Models Understand?
William Merrill, Yoav Goldberg, Roy Schwartz +1
Language models trained on billions of tokens have recently led to unprecedented results on many NLP tasks. This success raises the question of whether, in principle, a system can…
Back to Square One: Artifact Detection, Training and Commonsense Disentanglement in the Winograd Schema
Yanai Elazar, Hongming Zhang, Yoav Goldberg +1
The Winograd Schema (WS) has been proposed as a test for measuring commonsense capabilities of models. Recently, pre-trained language model-based approaches have boosted performanc…
Contrastive Explanations for Model Interpretability
Alon Jacovi, Swabha Swayamdipta, Shauli Ravfogel +3
Contrastive explanations clarify why an event occurred in contrast to another. They are more inherently intuitive to humans to both produce and comprehend. We propose a methodology…
Bootstrapping Relation Extractors using Syntactic Search by Examples
Matan Eyal, Asaf Amrami, Hillel Taub-Tabib +1
The advent of neural-networks in NLP brought with it substantial improvements in supervised relation extraction. However, obtaining a sufficient quantity of training data remains a…
Measuring and Improving Consistency in Pretrained Language Models
Yanai Elazar, Nora Kassner, Shauli Ravfogel +4
Consistency of a model -- that is, the invariance of its behavior under meaning-preserving alternations in its input -- is a highly desirable property in natural language processin…
A simple geometric proof for the benefit of depth in ReLU networks
Asaf Amrami, Yoav Goldberg
We present a simple proof for the benefit of depth in multi-layer feedforward network with rectified activation ("depth separation"). Specifically we present a sequence of classifi…