45 citations · 144 across the 6 of their papers we have counts for
7 papers
Jamba: A Hybrid Transformer-Mamba Language Model
Opher Lieber, Barak Lenz, Hofit Bata +19
We present Jamba, a new base large language model based on a novel hybrid Transformer-Mamba mixture-of-experts (MoE) architecture. Specifically, Jamba interleaves blocks of Transfo…
Human or Not? A Gamified Approach to the Turing Test
Daniel Jannai, Amos Meron, Barak Lenz +2
We present "Human or Not?", an online game inspired by the Turing test, that measures the capability of AI chatbots to mimic humans in dialog, and of humans to tell bots from other…
MRKL Systems: A modular, neuro-symbolic architecture that combines large language models, external knowledge sources and discrete reasoning
Ehud Karpas, Omri Abend, Yonatan Belinkov +14
Huge language models (LMs) have ushered in a new era for AI, serving as a gateway to natural-language-based knowledge tasks. Although an essential element of modern AI, LMs are als…
Standing on the Shoulders of Giant Frozen Language Models
Yoav Levine, Itay Dalmedigos, Ori Ram +10
Huge pretrained language models (LMs) have demonstrated surprisingly good zero-shot capabilities on a wide variety of tasks. This gives rise to the appealing vision of a single, ve…
Exemplar Guided Active Learning
Jason Hartford, Kevin Leyton-Brown, Hadas Raviv +3
We consider the problem of wisely using a limited budget to label a small subset of a large unlabeled dataset. We are motivated by the NLP problem of word sense disambiguation. For…
PMI-Masking: Principled masking of correlated spans
Yoav Levine, Barak Lenz, Opher Lieber +4
Masking tokens uniformly at random constitutes a common flaw in the pretraining of Masked Language Models (MLMs) such as BERT. We show that such uniform masking allows an MLM to mi…
SenseBERT: Driving Some Sense into BERT
Yoav Levine, Barak Lenz, Or Dagan +6
The ability to learn from large unlabeled corpora has allowed neural language models to advance the frontier in natural language understanding. However, existing self-supervision t…