8 citations · 15 across the 4 of their papers we have counts for
5 papers
Jamba-1.5: Hybrid Transformer-Mamba Models at Scale
Jamba Team, Barak Lenz, Alan Arazi +58
We present Jamba-1.5, new instruction-tuned large language models based on our Jamba architecture. Jamba is a hybrid Transformer-Mamba mixture of experts architecture, providing hi…
Generating Benchmarks for Factuality Evaluation of Language Models
Dor Muhlgay, Ori Ram, Inbal Magar +7
Before deploying a language model (LM) within a given domain, it is important to measure its tendency to generate factually incorrect information in that domain. Existing methods f…
Parallel Context Windows for Large Language Models
Nir Ratner, Yoav Levine, Yonatan Belinkov +7
When applied to processing long text, Large Language Models (LLMs) are limited by their context window. Existing efforts to address this limitation involve training specialized arc…
Fewer Errors, but More Stereotypes? The Effect of Model Size on Gender Bias
Yarden Tal, Inbal Magar, Roy Schwartz
The size of pretrained models is increasing, and so is their performance on a variety of NLP tasks. However, as their memorization capacity grows, they might pick up more social bi…
Data Contamination: From Memorization to Exploitation
Inbal Magar, Roy Schwartz
Pretrained language models are typically trained on massive web-based datasets, which are often "contaminated" with downstream test sets. It is not clear to what extent models expl…