2 papers
cs.CL2024
Description-Based Text Similarity
Shauli Ravfogel, Valentina Pyatkin, Amir DN Cohen +2
Identifying texts with a given semantics is central for many information seeking scenarios. Similarity search over vector embeddings appear to be central to this ability, yet the s…
cs.CL2024
Jamba: A Hybrid Transformer-Mamba Language Model
Opher Lieber, Barak Lenz, Hofit Bata +19
We present Jamba, a new base large language model based on a novel hybrid Transformer-Mamba mixture-of-experts (MoE) architecture. Specifically, Jamba interleaves blocks of Transfo…