45 citations · 83 across the 4 of their papers we have counts for
5 papers
MRKL Systems: A modular, neuro-symbolic architecture that combines large language models, external knowledge sources and discrete reasoning
Ehud Karpas, Omri Abend, Yonatan Belinkov +14
Huge language models (LMs) have ushered in a new era for AI, serving as a gateway to natural-language-based knowledge tasks. Although an essential element of modern AI, LMs are als…
Standing on the Shoulders of Giant Frozen Language Models
Yoav Levine, Itay Dalmedigos, Ori Ram +10
Huge pretrained language models (LMs) have demonstrated surprisingly good zero-shot capabilities on a wide variety of tasks. This gives rise to the appealing vision of a single, ve…
Exemplar Guided Active Learning
Jason Hartford, Kevin Leyton-Brown, Hadas Raviv +3
We consider the problem of wisely using a limited budget to label a small subset of a large unlabeled dataset. We are motivated by the NLP problem of word sense disambiguation. For…
PMI-Masking: Principled masking of correlated spans
Yoav Levine, Barak Lenz, Opher Lieber +4
Masking tokens uniformly at random constitutes a common flaw in the pretraining of Masked Language Models (MLMs) such as BERT. We show that such uniform masking allows an MLM to mi…
SenseBERT: Driving Some Sense into BERT
Yoav Levine, Barak Lenz, Or Dagan +6
The ability to learn from large unlabeled corpora has allowed neural language models to advance the frontier in natural language understanding. However, existing self-supervision t…