6 citations · 21 across the 21 of their papers we have counts for
4 papers · 1 filter
Power-Softmax: Towards Secure LLM Inference over Encrypted Data
Itamar Zimerman, Allon Adir, Ehud Aharoni +7
Modern cryptographic methods for implementing privacy-preserving LLMs such as \gls{HE} require the LLMs to have a polynomial form. Forming such a representation is challenging beca…
DeciMamba: Exploring the Length Extrapolation Potential of Mamba
Assaf Ben-Kish, Itamar Zimerman, Shady Abu-Hussein +4
Long-range sequence processing poses a significant challenge for Transformers due to their quadratic complexity in input length. A promising alternative is Mamba, which demonstrate…
Explaining Modern Gated-Linear RNNs via a Unified Implicit Attention Formulation
Itamar Zimerman, Ameen Ali, Lior Wolf
Recent advances in efficient sequence modeling have led to attention-free layers, such as Mamba, RWKV, and various gated RNNs, all featuring sub-quadratic complexity in sequence le…
The Hidden Attention of Mamba Models
Ameen Ali, Itamar Zimerman, Lior Wolf
The Mamba layer offers an efficient selective state space model (SSM) that is highly effective in modeling multiple domains, including NLP, long-range sequence processing, and comp…